Linux-KVM (Kernel-based Virtual Machine) Explained
The KVM API
Introduction
In this article, we will go through the KVM (Kernel-based Virtual Machine) API. KVM is a Linux kernel subsystem that can be used to start a virtual machine that uses hardware virtualization technologies such as VT-x of Intel or AMD-V. Hardware virtualization technologies give way better performance than a full emulation of a virtual machine.
The KVM module is usually used as part of a hypervisor such as QEMU or kvmtool (a lesser-known alternative to QEMU). A hypervisor is the software layer responsible for managing the virtual machine: starting it; scheduling it; emulating the hardware, and so on. When QEMU uses KVM to start a virtual machine, the couple is called: QEMU-KVM.
The API
KVM is an event-driven module; as is the case for most of the Linux kernel subsystems. The events are triggered by the user-space (QEMU) through a well-defined API (Application Programming Interface). This API has no system call for itself and relies heavily on the ioctl (input/output control) system call.
An ioctl requires two main arguments: a file descriptor and a request number. There is also a list of optional arguments that depends on the request number.
int ioctl(int fd, unsigned long request, …);
There are 4 types of file descriptors relating to KVM: system file descriptor, VM file descriptor, vcpu file descriptor, and device file descriptor. Thus there are 4 types of ioctl of the KVM API interface: system, VM, vcpu, and device ioctls. To obtain a system file descriptor, we simply need to open the file “/dev/kvm” (Note: this file is present only when Linux kernel is compiled with the KVM module):
kvm_fd = open(“/dev/kvm”, O_RDWR);
The kvm_fd can be used to configure the KVM module. But most importantly it can be used to create the VM file descriptor, that will represent a virtual machine. This is done by executing:
vm_fd = ioctl(kvm_fd, KVM_CREATE_VM, vm_type);
The vm_fd can be used to configure the virtual machine that it represents. It can also be used to create a number of vcpu within the virtual machine or a device. This is done by executing the following ioctls:
vcpu_fd = ioctl(vm_fd, KVM_CREATE_VCPU, vcpu_id);
device_fd = ioctl(vm_fd, KVM_CREATE_VCPU, kvm_device_structure);As both for both previous file descriptors, the vcpu_fd one can be used to configure the vcpu that it represents. This apply similarly to the device file descriptor. An example of an ioctl is the following:
ret = ioctl(vcpu_fd, KVM_RUN);
The above ioctl launch the vcpu that called it to run within the virtual machine.
Putting it all together
//create kvm_fd
kvm_fd = open(“/dev/kvm”, O_RDWR);
//kvm_fd used to create vm_fd
vm_fd = ioctl(kvm_fd, KVM_CREATE_VM, …);
//vm_fd used to creates vcpu_fd
vcpu_fd = ioctl(vm_fd, KVM_CREATE_VCPU, …);
//vm_fd creates device_fd
device_fd = ioctl(vm_fd, KVM_CREATE_VCPU, …);
//run the calling VCPU in the virtual machine
ret = ioctl(vcpu_fd, KVM_RUN);
Examples from QEMU
Together we will explore the source code of QEMU to find the calls to the KVM API. Please find th used source code at: https://github.com/qemu/qemu/tree/stable-6.1.We will be using mostly a grep on the source code. To find the location of the creation of the kvm_fd we use:
$ grep ‘/dev/kvm’ -rI *
This command give us few lines of output. The most important line is the one pointing to the file “accel/kvm/kvm-all.c”. The file can be accessed directly through this link. At line 2335, we find this
s->fd = qemu_open_old(“/dev/kvm”, O_RDWR);
The file “accel/kvm/kvm-all.c” also contains the line of codes that creates the VM at line 2381, that create the vcpu at line 451, and that creates the device at line 3419. The vcpu ioctl “KVM_RUN” can also be found at line 2843.
Conclusion
That was a brief description of the KVM API. Feel free to take a look at the Linux-kernel documentation for a more in-depth description of the API (https://www.kernel.org/doc/Documentation/virtual/kvm/api.txt).
Finally, dropping a comment or to liking this article will be greatly appreciated 🙂.
