How to Enable Nested Virtualization in KVM on RHEL 8 / Rocky Linux 8

In this post, we will show you how to enable nested virtualization in KVM on RHEL 8 / Rocky Linux 8.

Nested virtualization in KVM (Kernel-based Virtual Machine) is a feature that allows you to run virtual machines (VMs) inside other virtual machines. This means that you can create a virtual environment within a virtual machine and run another virtual machine inside it.

Nested virtualization is particularly useful for testing and development scenarios, where you may want to create multiple VMs with different configurations or operating systems without the need for separate physical hardware. For example, you can use nested virtualization to test the compatibility of an application running on multiple versions of an operating system.

Prerequisites

• Pre Install KVM on RHEL 8 / Rocky Linux 8
• Sudo User or Root User access

I am assuming you have already configured KVM hypervisor. In case you have not familiar on how to install and configure KVM hypervisor, then refer the following article

• How to Install and Configure KVM on RHEL 8 / CentOS 8

Without further ado, let’s jump into actual steps.

Enabled KVM Nested Virtualization

Verify whether nested virtualization is enabled or not on your KVM host

For Intel based Processors run the following cat command,

[root@kvm-hypervisor ~]# cat /sys/module/kvm_intel/parameters/nested
N
[root@kvm-hypervisor ~]#

For AMD based Processors run the command,

[root@kvm-hypervisor ~]# cat /sys/module/kvm_amd/parameters/nested
N
[root@kvm-hypervisor ~]#

In the output above ‘N’ indicates that Nested virtualization is disabled. If we get the output as ‘Y’ then it indicates that nested virtualization is enabled on your host.

Now to enable nested virtualization, create a file with the name “/etc/modprobe.d/kvm-nested.conf” with the following content.

[root@kvm-hypervisor ~]# vi /etc/modprobe.d/kvm-nested.conf
options kvm-intel nested=1
options kvm-intel enable_shadow_vmcs=1
options kvm-intel enable_apicv=1
options kvm-intel ept=1

Save & exit the file

Now remove ‘kvm_intel‘ module and then add the same module with modprobe command. Before removing the module, make sure VMs are shutdown otherwise we will get following error message “modprobe: FATAL: Module kvm_intel is in use”

[root@kvm-hypervisor ~]# modprobe -r kvm_intel
[root@kvm-hypervisor ~]# modprobe -a kvm_intel
[root@kvm-hypervisor ~]#

Now verify whether nested virtualization feature enabled or not, run

[root@kvm-hypervisor ~]# cat /sys/module/kvm_intel/parameters/nested
Y
[root@kvm-hypervisor ~]#

For AMD based systems, run the below commands,

[root@kvm-hypervisor ~]# rmmod kvm-amd
[root@kvm-hypervisor ~]# echo 'options kvm-amd nested=1'>>/etc/modprobe.d/dist.conf
[root@kvm-hypervisor ~]# modprobe kvm-amd

Test KVM Nested Virtualization

Let’s suppose we have a VM with name “director” on KVM hypervisor on which I have enabled nested virtualization. Before testing, make sure CPU mode for the VM is either as “host-model” or “host-passthrough” , to check cpu mode of a virtual machine use either Virt-Manager GUI or virsh edit command.

# virsh edit director

For the new VMs for which you want to use nested virtualization, check the “Copy host CPU configuration” option under the CPU settings from virt-manager.

Now login to the director VM and run lscpu and lsmod command

[root@kvm-hypervisor ~]# ssh 192.168.126.1 -l root
[email protected]'s password:
Last login: Sun Dec 10 07:05:59 2017 from 192.168.126.254
[root@director ~]# lsmod | grep kvm
kvm_intel             170200  0
kvm                   566604  1 kvm_intel
irqbypass              13503  1 kvm
[root@director ~]#
[root@director ~]# lscpu

Let’s try creating a virtual machine either from virtual manager GUI or virt-install inside the director vm, in my case i am using virt-install command

[root@director ~]# virt-install  -n Nested-VM  --description "Test Nested VM" \
 --os-type=Linux  --os-variant=rhel7  --ram=2048  --vcpus=2 --disk \
 path=/var/lib/libvirt/images/nestedvm.img,bus=virtio,size=10 --graphics \
 none --location /var/lib/libvirt/images/CentOS-7-x86_64-DVD-1511.iso\
 --extra-args console=ttyS0
Starting install...
Retrieving file .treeinfo...                            | 1.1 kB  00:00:00
Retrieving file vmlinuz...                              | 4.9 MB  00:00:00
Retrieving file initrd.img...                           |  37 MB  00:00:00
Allocating 'nestedvm.img'                               |  10 GB  00:00:00
Connected to domain Nested-VM
Escape character is ^]
[    0.000000] Initializing cgroup subsys cpuset
[    0.000000] Initializing cgroup subsys cpu
[    0.000000] Initializing cgroup subsys cpuacct
[    0.000000] Linux version 3.10.0-327.el7.x86_64 
………………………………………………

This confirms that nested virtualization has been enabled successfully as we are able to create virtual machine inside a virtual machine.

That’s all from this post, kindly do post your queries and feedback in below comments section.