QEMU: Dive into QEMU

1.   Major components of QEMU

• TCG (Tiny Code Generator)
• Address Translation
• Device emulation

1.1    Tiny Code Generator:

                                                    This is basically a run time translator that converts target CPU instructions to host CPU instructions. Febrice Bellard (founder of QEMU) has written a paper about this dynamic translation. In short it translates the target CPU instructions to host CPU instructions that work with emulated memory space. There is a tcg folder in QEMU that contains most of target specific code generator’s code. Also there is a tcg.c file in main directory that contains generic code.

1.2    Memory Address Translation:

QEMU has a soft MMU model to translate target virtual addresses to host virtual addresses. SofMMU provides two types of mapping:

• Mapping between Guest virtual address and host virtual addresses
• Mapping between Guest virtual address and registered I/O functions for that device.

Mapping between Guest and host virtual addresses:

QEMU does not perform the address translation process on addresses i.e. it does not first translate target virtual address to target physical address then target physical address to host physical address and then host physical address to host virtual address. Rather QEMU uses address translation cache that does direct translation from target virtual address to host virtual address. This translation cache stores Addend for guest addresses. This Addend is actually offset of host virtual address from guest virtual address. This means

Host virtual address = Guest virtual address + Addend

Cache used for this is tlb_table. When translating the guest virtual address to host virtual address, it will search this TLB table firstly. If there is an entry in the table, then QEMU can add this offset to guest virtual address to get the host virtual address directly. You can read more about it from http://vm-kernel.org/blog/2009/07/10/qemu-internal-part-2-softmmu/

Mapping between Guest virtual address and registered I/O functions:

Cache used for memory mapped I/O accesses is iotlb. Iotlb is used to translate addresses to index of io_mem_write and io_mem_read arrays. Iotlb is defined in cpu_defs.h as

target_phys_addr_t iotlb[NB_MMU_MODES][CPU_TLB_SIZE];

io_mem_write and io_mem_read are two dimensional arrays of function pointers. These take index from tlb and invoke its respective function.

 All these Mappings are done in Softmmu_template.h and Softmmu_header.h. Softmmu header contains generic functions for load, store, load and store instructions.

2.   QEMU Peripheral Simulations:

2.1. Available simulations of peripherals in QEMU

• - i440FX host PCI bridge and PIIX3 PCI to ISA bridge
• - Cirrus CLGD 5446 PCI VGA card or dummy VGA card with Bochs VESA extensions (hardware level, including all non standard modes).
• - PS/2 mouse and keyboard
• - 2 PCI IDE interfaces with hard disk and CD-ROM support
• - Floppy disk
• - PCI and ISA network adapters
• - Serial ports
• - Creative SoundBlaster 16 sound card
• - ENSONIQ AudioPCI ES1370 sound card
• - Intel 82801AA AC97 Audio compatible sound card
• - Adlib(OPL2) - Yamaha YM3812 compatible chip
• - Gravis Ultrasound GF1 sound card
• - CS4231A compatible sound card
• - PCI UHCI USB controller and a virtual USB hub.

2.2. Virtual Serial Port:

Set using –serial switch and serial output can be redirected to different devices using different options

1. COMn(windows only): Redirect serial I/O to host serial port
2. file:filename: Write output to some file….Works only for redirection of serial output and no serial data can be read.
3. none: disable all serial ports
4. vc: redirect to virtual console. Virtual console is in QEMU window and user can switch to it using Ctrl-Alt-3 option.

2.3. Control Flow in Serial simulation:

2.4. How serial port emulation works:

•         The generic part of UART emulation is implemented in qemu-char.c file.

•         Controller specific part of PL011 UART controller is in hw/pl011.c file. This file also contains MMIO read/write register handlers.

•         Register handlers in pl011.c calls generic read/writefunctions from qemu-char.c.

•         For example, pl011_read function calls qemu_chr_accept_input from qemu_char.c. Similarly, pl011_write in pl011.c calls qemu_chr_write from qemu-char.c.

•         The read/write handler functions pl011_read and pl011_write handle memory access to all the registers of the device. They implement a switch/case for the address to select a particular register.

2.4.1.  Layers in serial Port Emulation:

2.4.1.1.          SOFTMMU Layer:

                     Its code is in softmmu_template.h/softmmu_header.h. softmmu is used to accelerate the process of finding the mapping between guest physical address and host virtual address and the mapping between guest I/O region and qemu I/O emulation functions.

2.4.1.2.          Controller Specific Handlers:

                                               Every controller will register handlers for its memory space and these handler functions will be invoked by sofmmu.

PL011 UART Contoller Emulation: Its source code is in PL011.c. It contains PL011 UART controller's emulation inculding its read/write handler functions.

2.4.1.3.      Generic Read/Write Functions:

                                                                Its source code is in qemu-char.c. QEMU has a generic layer that provides generic functions for reading and writing serial data from/to different modes e.g. host serial port, stdio, QEMU's virtual console etc.

2.4.1.4.      Virtual Console Mode:

                                               Its source code is in console.c. QEMU has a command parser that parses command used to launch qemu vm. This parser sets mode of serial communication depending upon the switch. Generic function invokes these mode specific functions using function pointer which is set after command parsing.

2.4.2.  EXAMPLE Nucleus SHELL Demo:

2.4.3.  Control Flow in SoftMMU:

2.4.3.1.          Tiny Code Generator (TCG):

                                                It translates target instructions to host instructions. In QEMU every emulated target has a translation file (translate.c) that contains the complete source code of translator. This translator translates emulated target’s instructions to host machine i.e. x86. Below is pictorial representation of it.

Translator Details:

1. DissAssembler

ARM translator has a disassembler function in it that translates machine language instructions to assembly language. It is disas_arm_insn function.

2.4.3.2.          TLBs: Maps target virtual address to host virtual address or Read/write handler (in case of Memory Mapped devices).

2.4.4.  Overall Control Flow in QEMU:

•         User gives command to launch qemu.

•         QEMU’s command parser mechanism parses given command and registers devices accordingly.

•         After this initialize function of target board (given in command) is invoked, default target is integratorcp for ARM.

•         Target initialization function creates devices. E.g. PL011 UART controller and also pass its memory address to creation function.

•         It then calls pl011_init function in pl011.c to initialize pl011 controller and passes memory address. It then register handlers for pl011 controller’s memory region.

2.5. Emulation for Linux based USB Host side applications:

                                                                               

User is developing a Linux based USB host application for some embedded target with host controller (EHCI/OHCI or UHCI) and he wants to test his application

QEMU:

            Features:

                        Allows Host controller emulation for 3 platforms.

·         PC System has a UHCI controller

·         MIPS/malta has a PIIX4 PCI/USB/SMbus controller

·         ARM Architecture ones:

o    ARM Versatile baseboard with a  PCI OHCI controller

o    ARM RealView Emulation/Platform with a  PCI OHCI controller

o    XScale-based clamshell PDA models with an On chip OHCI controller

o    Nokia N800 and N810 internet tablets

            Limitations:

·         Allows to test with physical USB devices plugged in host USB port but not for streaming devices (using Isochronous transfer). It means if user is developing a host side application with any Isochronous transfer based USB device.

·         Physical USB devices can be used in QEMU only in case of LINUX host.

                                   

2.6. Emulation for Linux based USB Function side applications:

User is developing an Embedded linux based USB device and wants to test it on his host machine without embedded target board by virtually plugging it in to host PC’s USB controller port.

Example: User is developing an embedded linux based mass storage device and wants to test his device side code.

QEMU: Provides emulation of USB devices which can be plugged with QEMU platform but I’ve not found any QEMU based solution that allows me to run an EL based USB device under QEMU and virtually plug that device in to host PC’s USB port..

Currently I am exploring dummy HCD in Linux, a model used by Openmoko for USB  and  USB/IP.

Below is a pdf document explaining OpenMoko’s emulator and USB settings.

http://www.csie.mcu.edu.tw/~wangch/mvs/openmoko_intro.pdf

This emulator allows users to connect the virtual, emulated Neo1973 ( OpenMoko’s hardware platform) to the Linux PC on which the emulator is running, and work with it as if a real Neo1973 was plugged into the computer's USB port. How if we take this idea for Android and other EL distributions based USB devices?

2.6.1.  Android Emulator:

1.     Device side applications:

                                                Developing an application that allows user to use Android cell phone’s camera as a Webcam on host machine.

           

a.     Emulator Status:

No support for USB device side application emulation.

2.     Host side Applications:

The official kernel provided for Android phones does not have any USB host capabilities enabled. Android for mobile devices does not support USB host feature. But if we talk about Android for embedded platforms then it must have this feature enabled in the kernel.

a.     Emulator Status:

No support for USB host side application emulation.

3.     OTG:

The official kernel provided for Android phones does not have any USB OTG capabilities enabled. But if we talk about Android for embedded platforms then it must have this feature enabled in the kernel.

a.     Emulator Status:

                                                No support for USB OTG controllers emulation.

2.7. QEMU’s Network Emulation:

QEMU has a number of ways to set up networking for its guest applications. QEMU can simulate various network cards and can connect them to an arbitrary number of Virtual Local Area Networks (VLANs).

2.7.1.Virtual LAN:

A VLAN is a network switch running in the context of a qemu process. It can be symbolised as a virtual connection between several network devices. These devices can be for example QEMU virtual Ethernet cards or virtual Host ethernet devices (TAP devices). There can be multiple vlans in each qemu VM process. Multiple interfaces can be connected to each vlan. Anything sent via one of the interfaces on a vlan is received by all the other interfaces on that vlan. One example of interface on vlan is NIC card. Any frames sent by the guest OS over that nic will appear on the vlan which the nic is a member of. Any frames sent to the vlan from other interfaces will be received by the guest OS via the nic.

VLAN is QEMU VM process specific and it is not something that can be used to connect multiple QEMU instances. Below is pictorial representation of it (Got most of these from a website)

         +-----------+

         |   Guest   |

         |    OS     |

         |   +---+   |

         |   |NIC|   |

         +---+-+-+---+

               ^           +--------+

               |           |        +<------> Other IF (say another NIC in QEMU)

               +---------->+  VLAN  |

                           |        +<------> Other IF (host TAP device)

                           +--------+

2.7.2. Default User mode Network stack:

By using the option ‘-net user’, QEMU uses a completely user mode network stack (you don’t need root privilege to use the virtual network). The virtual network configuration is the following:

+-----------+          |   Guest   |          |    OS     |          |   +---+   |          |   |NIC|   |          +---+-+-+---+                ^           +--------+                |           |        +                +---------->+  VLAN  |<------> Firewall/DHCP server <------> Internet                            |        +         | (10.0.2.2)                            +--------+         |                                                -----> DNS Server(10.0.2.3)                                               |                                                -----> SMB Server (10.0.2.4)

The QEMU VM behaves as if it was behind a firewall which blocks all incoming connections. You can use a DHCP client to automatically configure the network in the QEMU VM. This user mode network stack is part of QEMU and there will be no other process running for DHCP and DNS servers. The DHCP server assigns addresses to the hosts starting from 10.0.2.15.

2.7.3.      Port Redirection:

                                                     As QEMU process acts as a firewall between Guest OS and host network, it does not support protocols other than TCP and UDP which means ping and ICMP can not be used from host for Guest network. For this QEMU provides an option of port redirection using which TCP or UDP connections on host can be redirected to guest OS. For example user can redirect telnet connections from host port 5555 to guest port using following command:

qemu -net user,hostfwd=tcp:5555::23

2.7.4.Connecting Multiple QEMU VMs (VLANs):

2.7.4.1.    Using Sockets:

Every QEMU process with network card emulation has at least one virtual LAN. Multiple VLANs can be connected together using socket.

      +-----------+                                      +-----------+

      |   Guest   |                                      |   Guest   |

      |     A     |                                      |     B     |

      |   +---+   |                                      |   +---+   |

      |   |NIC|   |                                      |   |NIC|   |

      +---+-+-+---+                                      +---+-+-+---+

            ^       +--------+                +--------+       ^

            |       |        |                |        |       |

            +------>+ VLAN 0 +<--> Socket <-->+ VLAN 2 +<------+

                    |        |                |        |

                    +--------+                +--------+

In this case one QEMU VM process connects to a socket in another QEMU VM.

2.7.4.2.  Using Host TAP devices:

                                                      I had to spend quite some time to learn about host TUN/TAP devices and I must say QEMU documentation does not provide any good explanation of it. TUN/TAP provides virtual Ethernet driver, which instead of receiving packets from a physical media, receives them from user space program and instead of sending packets via physical media sends them to the user space program. This is very similar to Nucleus SimTest, a simulated device (simulation model) and a “virtual Ethernet driver” for that device.

So in case of QEMU VM, it will be running as a user space program that can read/write packets from/to host TAP device. Configuring TAP devices is a bit difficult. It requires installing VPN on host and then adding bridge between host TAP device and physical Ethernet but you can easily find shell script that does this for you.

VDE (Virtual Distributed Ethernet) is a user program which can obtain a TAP device and allow a number of other programs(QEMU VMs) to connect to it and bridge those connections to the TAP device. There is a separate switch in QEMU for VDE.

    +-----------+                           +-----------+

    |   Guest   |                           |   Guest   |

    |     A     |                           |     B     |

    |   +---+   |                           |   +---+   |

    |   |NIC|   |                           |   |NIC|   |

    +---+-+-+---+                           +---+-+-+---+

          ^       +------+         +------+       ^

          |       |      | +-----+ |      |       |

          +------>+ VLAN +-+ VDE +-+ VLAN +<------+

                  |      | +--+--+ |      |

                  +------+    |    +------+

                              |

                          +---+-----+  +--------+

                          | Kernel  |  |  TAP   |

                          | TUN/TAP +--+ Device |

                          | Driver  |  | (qtap) |

                          +---------+  +--------+

2.7.4.3.    Using Port Redirection:

                                                             Android emulator uses this port redirection feature for inter connecting multiple instances of emulator.  

Based on all this, I assume that QEMU provides different options for emulating distributed multi target applications. Although it is difficult to setup and needs some effort but IT IS FREE.

2.8. FILE System Emulation:

I’ve been looking in to QEMU VM’s ability to communicate with host file system. Actually there are two modes of QEMU

1. User Space emulation
2. System emulation

2.8.1.     User space emulation:

                                      It allows running a Linux process (compiled for some other architecture) on host PC. E.g. I’ve developed an embedded linux application for ARM integrator. I can build this application on my host using cross development toos/IDEs. QEMU allows me to run this ARM binary in my host machine using QEMU’s user space emulation. The emulated Linux process is just like a host process and it call access host file system as well.

2.8.2.     System emulation:

                                It emulates a full system including processor and peripherals. This mode is similar to other VMs only thing is that it also allows emulation of non PC (embedded) targets. It provides emulation for disk as well that allows creation of virtual disk images of different formats.

2.8.3.  Transferring files to and from the host

1.     The emulator and file system are set up to automatically configure a virtual Ethernet interface in the virtual machine with an internal IP. Through that virtual network interface, the emulator is set up to enable transferring of files to and from the host machine file system using the TFTP protocol. I mentioned about this user mode network emulation in my last email about network emulation in QEMU.

I downloaded ARM Linux 2.6 kernel image and a RAM disk file system image from QEMU’s website. I then started my QEMU VM (ARM integrator) using following command:

qemu-system-arm –kernel zImage.integrator –initrd arm_root.img –tftp /

This command starts QEMU in system emulation mode emulating an ARM integrator platform (default platform for qemu-system-arm). It boots into kernel image zImage.integrator and loads it into VM’s RAM i.e. arm_root.img.

–tftp / instructs QEMU to make entire host root file system available for access via TFTP from VM.

I can now access my host file system using following command on guest Linux terminal:

Tftp –g –r /usr/QEMU_Learning/Test_Documents/test.txt –l guest_test.txt 10.0.2.2

This will read test.txt from my host file system and will copy it to guest system.

2.     SSH in QEMU VM:

     Copy it from emails.