#8208 communishift: allow exposing /dev/kvm to containers
Closed: Fixed 6 months ago by kevin. Opened a year ago by dustymabe.

I have a project where I'd like to expose /dev/kvm to the running containers so that I can use unprivileged qemu-kvm to achieve some tasks. I've been told in OCP 4.1 land that this is achievable by using a kvm kubernetes device plugin. The docs for that are here. There is a daemonset we can use to set it up and then we can request /dev/kvm in our pod manifests by including:

spec:
  containers:
  - name: demo
    ...
    resources:
      requests:
              devices.kubevirt.io/kvm: "1"
      limits:
              devices.kubevirt.io/kvm: "1"

Can we investigate adding the ability to expose /dev/kvm to our containers?


cc @fabiand - in case he has any new information here beyond what is documented on that page.

Metadata Update from @kevin:
- Issue priority set to: Waiting on Assignee (was: Needs Review)

a year ago

Yes, KubeVirt is using a daemonset with a DP to expose kvm as a resource, and provide it to (unprivileged) containers.
The issue is that this DP is part of our node agent (virt-handler) thus not usable standalone.

What's your use-case for running a VM in a containre?
Aka why can't you use KubeVirt?

I can't speak for @dustymabe but our support tooling is not designed for KubeVirt and we'd rather spend our time elsewhere. The other users of that support tooling end up using privileged containers and are not affected in the same way.

Yes, KubeVirt is using a daemonset with a DP to expose kvm as a resource, and provide it to (unprivileged) containers.
The issue is that this DP is part of our node agent (virt-handler) thus not usable standalone.
What's your use-case for running a VM in a containre?
Aka why can't you use KubeVirt?

We're basically running unprivileged qemu commands (some invoked directly and some indirectly via libguestfs) to build Fedora CoreOS (via coreos-assembler) in an OpenShift environment. Previously we have been using the oci-kvm-hook rpm for this but now in the CoreOS world (where you can't easily just add an rpm to the host) we're looking for the new way of doing things. I was pointed at this doc for that.

So if we just wanted to be able to create pods that have /dev/kvm mounted in is that doc the best way to do it these days?

Yes. It's a litlte bit outdated but should work.
The other option is to use a privileged pod or hostPath

Metadata Update from @cverna:
- Issue tagged with: backlog

a year ago

Metadata Update from @mizdebsk:
- Issue tagged with: communishift

10 months ago

This should be as simple as:

#!/bin/bash
set -eux
if ! oc get --namespace=default ds/device-plugin-kvm &>/dev/null; then                                                                                                                                                                       
    oc create --namespace=default -f https://raw.githubusercontent.com/kubevirt/kubernetes-device-plugins/master/manifests/kvm-ds.yml                                                                                                        
fi

daemonset.apps/device-plugin-kvm created

Give it a test and see if it's working.

It's working like a charm!

$ oc create -f https://raw.githubusercontent.com/kubevirt/kubernetes-device-plugins/master/examples/kvm-consumer.yml
pod/kvm-consumer created
$ oc rsh pod/kvm-consumer ls -l /dev/kvm
crw-rw-rw-    1 root     36         10, 232 Feb 14 02:59 /dev/kvm

I think the only remaining thing to do is to try to codify this so if/when we rebuild communishift it gets re-applied.

Metadata Update from @kevin:
- Issue close_status updated to: Fixed
- Issue status updated to: Closed (was: Open)

6 months ago

I think the only remaining thing to do is to try to codify this so if/when we rebuild communishift it gets re-applied.

Can we make sure this part got done so that we don't have to make the same request when communishift gets rebuilt?

Metadata Update from @kevin:
- Issue status updated to: Open (was: Closed)

6 months ago

Metadata Update from @kevin:
- Issue close_status updated to: Fixed
- Issue status updated to: Closed (was: Open)

6 months ago

Login to comment on this ticket.

Metadata