"How do we get there?"

"can I replace my VMs with containers? HOW!?"

Are they REALLY substitutes?
Is the one like the other?

Technology? Features? Feeling? Tools? Requirements?

⇝ It depends

Not yet?
"never"?

Yes

No

Cool

Replace?

Migration

If workloads can be moved to containers, then it's a migration

Convergence

If not, then we still want convergence

Yes

Migration Path?

Replace?

Doubled Infrastructure?

No

Management Plane

Storage

Network

…

Virtual Machines

Management Plane

Storage

Network

…

Containers

2x Infrastructure?

Containers
&
Virtual Machines

on the same infrastructure.

Management Plane

Storage

Network

…

Virtual Machines

Containers

Keep your VMs …

Management Plane

Storage

Network

…

Virtual Machines

Containers

… Transition what you need …

Management Plane

Storage

Network

…

Virtual Machines

Containers

… And stick to VMs as needed.

CRI? Can a pod be a VM?

• Add device details as annotations.
• Modify the container runtime on every node.
• Deal with the fact that there are two Pods when you do migrations.
• Implement as much functionality as possible from the Kubelet, since there is not way to distinguish from outside what your VM Pod supports, compared to a normal Pod.
• Are we talking about a VM Pod or a Pod?

Can VMs just be pods?

Or: Implementing a CRI

• Allows a proper Virtual Machine Specification
• We can ship KubeVirt as a pure add-on. No Node modifications are necessary.
• No matter, how much Pods are necessary to perform a migration, we have one single entrypoint to the Virtual Machine.
• Reuse all of the kubelet and Pod Spec functionality, by running a Virtual Machine inside the Pod
• Talk about VMs when they are VMs, talk about Pods when they are Pods.

A VM API?

No, an explicit virtualization API

VMs are different to pods, they require a dedicated API.

API
challenges

• Nothing Node-ish
• Add Kube concepts
• Add cluster concepts
• Imperative vs. Declarative
• How opinionated?

API challenges

• Feature wise comparable to domxml
• But
  • Some features are node specific - they need to be abstracted
  • Needs to be married with Kubernetes concepts (pv, networks)
  • Needs additional data for cluster-only features like scheduling
• Imperative vs. Declarative
  • Actions

kind: VirtualMachine

metadata:
  name: testvm

spec:
  domain:
    devices:
      type: PersistentVolumeClaim
      device: disk
      source:
        name: myVolumeClaim

  nodeSelector:
    cpu: fast

status:
  phase: Running

kind: Migration
metadata:
  generateName: my-migration
spec:
  nodeSelector:
    kubevirt.io/hostname: node1
  selector:
    name: testvm
status:
  phase: Succeeded

MIgrations

Backed by a controller:

• On object create, schedules a new Pod

• On successful Pod start, it triggers the migration

• At the end of the migration the object is moved to a final state

• Always one VirtualMachine object you reference

The objects Migration with VirtualMachine provide a consistent entry point to anything VirtualMachine related, like the Pod does for Kubernetes.

kind: VirtualMachine
metadata:
  name: testvm
spec:
  domain:
    devices:
      type: PersistentVolumeClaim
      device: disk
      source:
        name: myVolumeClaim
  nodeSelector:
    cpu: fast
status:
  phase: Running

KUBEVIRT API

We have the typical Pod like structure:

• Metadata section

• Specification section

• Typical Pod features like

  • nodeSelector

  • affinity

• Status section

Behind the scene a Pod is created, scheduled  and we make sure that the VM starts correctly inside.

# Typical Pod commands:

kubectl create -f mypodspec.yaml
kubectl delete mypod
kubectl exec mypod -it /bin/bash

# Typical VirtualMachine commands:

kubectl create -f myvmspec.yaml

kubectl plugin virt console testvm
kubectl plugin virt spice testvm

kubectl create -f myvmmigration.yaml

kubectl delete testvm

Workflow

kind: VirtualMachine
metadata:
  name: testvm
spec:
  domain:
    devices:
      graphics:
      - type: spice
      consoles:
      - type: pty


      

Typical Pod commands:

• kubectl create -f mypodspec.yaml

• kubectl delete mypod

• kubectl exec mypod -it /bin/bash

Typical VirtualMachine commands:

• kubectl create -f myvmspec.yaml

• kubectl delete testvm

• kubectl plugin virt console testvm

• kubectl plugin virt spice testvm

TYPICAL KUBECTL FEELING

Integration challenges

• qemu and libvirt in a Pod
• cgroups and namespaces
• Disk and networks
• VM migrations
• Functionality ownership

INTEGRATION Challenges

Properly integrate the VirtualMachine lifecycle in a Pod lifecycle.

• Disks
• Networking
• qemu with libvirt in a Pod
• cgroups and Namespaces
• Migrations on top of Kubernetes

$ minikube start --vm-driver kvm --network-plugin cni

$ git clone https://github.com/kubevirt/demo.git

$ cd demo

$ bash run-demo.sh

Try (with minikube)

$ bash run-demo.sh
# Deploying KubeVirt
...
vm "testvm" created
Waiting for KubeVirt to be ready ...
Waiting for KubeVirt to be ready ...
Waiting for KubeVirt to be ready ...
# KubeVirt is now ready. Try:
# $ kubectl get vms

$ kubectl get vms
NAME      KIND
testvm    VM.v1alpha1.kubevirt.io

$ kubectl get pods
NAME                                      READY     STATUS    RESTARTS   AGE
haproxy-723816479-wcblm                   1/1       Running   1          49s
iscsi-demo-target-tgtd-1270025779-nckbh   1/1       Running   0          48s
libvirt-8zj1k                             2/2       Running   0          48s
spice-proxy-3525077118-fswn9              1/1       Running   0          47s
virt-api-1956313626-t9rhj                 1/1       Running   0          46s
virt-controller-2251532855-tfm9f          1/1       Running   0          45s
virt-handler-s7g76                        1/1       Running   0          43s
virt-launcher-testvm-----q05vh            1/1       Running   0          38s
virt-manifest-1665692876-cs8wp            2/2       Running   0          42s

$ kubectl exec -it libvirt-8zj1k bash
Defaulting container name to libvirtd.
Use 'kubectl describe pod/libvirt-8zj1k' to see all of the containers in this pod.
# virsh list
 Id    Name                           State
----------------------------------------------------
 1     default_testvm                 running

# exit