A new model of device allocation in Kubernetes

Feruzjon Muyassarov @EST FI

Let's book a hotel

Device Plugins:

 

I need 1 room. The front desk just checks if a room is free and assigns you one. That could be next to the elevator, could have a broken AC, etc. You get a room, not a fit.

 

Device Plugins:

 

I need 1 room. The front desk just checks if a room is free and assigns you one. That could be next to the elevator, could have a broken AC, etc. You get a room, not a fit.

 

DRA:

I need 1 room. King bed, high floor, ocean view, etc.  The system matches you to a room with exactly those attributes.

Device Plugins:

 

I need 1 room. The front desk just checks if a room is free and assigns you one. That could be next to the elevator, could have a broken AC, etc. You get a room, not a fit.

 

DRA:

I need 1 room. King bed, high floor, ocean view, etc.  The system matches you to a room with exactly those attributes.

Device Plugins: integer, how many

DRA: how many + attributes

 

K8S VersionDRA Status
v1.26Alpha
v1.31Major redesign
v1.32Beta
v1.34GA
v1.36New features

Let's order some accelerators...

Before

resources:
  limits:
    nvidia.com/gpu: 1
  requests:
    nvidia.com/gpu: 1

Device plugins model ...

You ask for: nvidia.com/gpu: 1

You could not say:

  • with xyz capabilities
  • prepared in this way

 

Before

resources:
  limits:
    nvidia.com/gpu: 1
  requests:
    nvidia.com/gpu: 1

Device plugins model ...

You ask for: nvidia.com/gpu: 1

You could not say:

  • with xyz capabilities
  • prepared in this way

 

requests:
  - name: gpu
    deviceClassName: gpu
    selectors:
      - cel:
          expression: |
            device.model == "A100" &&
            device.memory >= quantity("40Gi")

After

DRA model...

Describe what you actually need...

You can ask for:

  • nvidia.com/gpu: 1
  • product ID: A100-SXM4-40GB 
  • memory: 40 GB

 

Driver developer

Cluster admin

Application Developer/DevOps

Roles

Driver developer

Cluster admin

Application Developer/DevOps

Roles

.. is someone who understands how a piece of hardware works, basically knows writing the software that lets to control and allocate that hardware.

Driver developer

Cluster admin

Application Developer/DevOps

Roles

.. is someone who understands how a piece of hardware works, basically knows writing the software that lets to control and allocate that hardware.

 

Decides:

  • what attributes of the hardware to expose to DRA
  • what interfaces to implement to configure the node resources on the fly

 

Driver developer

Cluster admin

App developer/DevOps

Roles

.. is someone who understands how a piece of hardware works, basically knows writing the software that lets to control and allocate that hardware.

Driver developer

Cluster admin

App developer/DevOps

Roles

.. is someone who knows the application needs and defines resource requirements for the their application

Let's play those roles.

Let's play those roles.

We’re not writing the driver today :)

  • Install a driver

 

 

 

 

Cluster admin

helm install dra-driver-cpu...
helm install dra-driver-cpu...
kubectl get resourceSlices
NAME                                         NODE                 DRIVER              POOL                 AGE
kind-control-plane-dra.cpu-9q2ls             kind-control-plane   dra.cpu             kind-control-plane   20h

Cluster admin

  • Install a driver

 

 

 

 

  • Analyze the resource(s)

 

 

 

 

  • Install a driver

 

 

 

 

ResourceSlices (menu)

attributes:
    cpuId: 0
    socketID: 0
    numaNodeID: 1
    smtEnabled: false
    FreqMHz: 3200
    threadCount: 8
 

attributes:
    cpuId: 1
    socketID: 0
    numaNodeID: 0
    smtEnabled: true
    FreqMHz: 2800
    threadCount: 8
 

attributes:
    cpuId: 1
    socketID: 0
    numaNodeID: 1
    smtEnabled: false
    FreqMHz: 3500
    threadCount: 8
 

 admin

reads

ResourceSlices (menu)

attributes:
    cpuId: 0
    socketID: 0
    numaNodeID: 1
    smtEnabled: false
    FreqMHz: 3200
    threadCount: 8
 

attributes:
    cpuId: 1
    socketID: 0
    numaNodeID: 0
    smtEnabled: true
    FreqMHz: 2800
    threadCount: 8
 

attributes:
    cpuId: 1
    socketID: 0
    numaNodeID: 1
    smtEnabled: false
    FreqMHz: 3500
    threadCount: 8
 

high

performance

  • server a
  • server b
  • server c
     

general
purpose

  • server g
  • server f

ARM

  • server a
  • server b
  • server f
     

Intel

  • server g
  • server h
  • server c
     

DeviceClasses (categorization)

 admin

reads

creates

high

performance

  • server a
  • server b
  • server c
     

general
purpose

  • server g
  • server f

ARM

  • server a
  • server b
  • server f
     

Intel

  • server g
  • server h
  • server c
     

DeviceClasses (categorization)


 apiVersion: resource.k8s.io/v1
 kind: DeviceClass
 metadata:
   name: high-performance
 spec:
   selectors:
     - cel:
         expression: >
           device.driver == 'intel.com' &&
           device.attributes["intel.com/family"].string == "Xeon" &&
           device.attributes["intel.com/cores"].int >= 32 &&
           device.attributes["intel.com/memoryGB"].int >= 256 &&
           device.attributes["intel.com/turboBoost"].bool == true

1. Devices are attached to the Nodes

2. Device driver is installed

3. ResourceSlices and DeviceClasses are created

Recap

...it is time for the DevOps to claim

resources

  • creates ResourceClaims (resources that a workload needs) and attaches to the workload (Pod)

DevOps/app dev

ResourceClaim == I would like to book a room with the sea view

  • creates ResourceClaims (resources that a workload needs) and attaches to the workload (Pod)

DevOps/app dev

apiVersion: resource.k8s.io/v1
kind: ResourceClaim
metadata:
  name: xeon-prioritized-claim
  namespace: default
spec:
  devices:
    requests:
      - name: cpu
        firstAvailable:
          # First choice: 5th gen Xeon, 64+ cores, AMX, high memory
          - name: xeon-gen5-large
            deviceClassName: high-performance
            count: 4                         
            selectors:
              - cel:
                  expression: |-
                    device.attributes["cpu.intel.com"].family == "Xeon" &&
                    device.attributes["cpu.intel.com"].generation >= 5 &&
                    device.attributes["cpu.intel.com"].cores >= 64 &&
                    device.attributes["cpu.intel.com"].memoryGB >= 512
          # Second choice: 4th gen Xeon, relaxed core/memory requirements
          - name: xeon-gen4-medium
            deviceClassName: high-performance
            count: 6                          
            selectors:
              - cel:
                  expression: |-
                    device.attributes["cpu.intel.com"].family == "Xeon" &&
                    device.attributes["cpu.intel.com"].generation >= 4 &&
                    device.attributes["cpu.intel.com"].cores >= 32 &&
                    device.attributes["cpu.intel.com"].memoryGB >= 256
  • creates ResourceClaims (resources that a workload needs) and attaches to the workload (Pod)

DevOps/app dev

apiVersion: resource.k8s.io/v1
kind: ResourceClaimTemplate
metadata:
  name: xeon-prioritized-claim-template
  namespace: default
spec:
  spec:
    devices:
      requests:
        - name: cpu
          firstAvailable:
            # First choice: 5th gen Xeon, 64+ cores, AMX, high memory
            - name: xeon-gen5-large
              deviceClassName: high-performance
              count: 1                          
              selectors:
                - cel:
                    expression: |-
                      device.attributes["cpu.intel.com"].family == "Xeon" &&
                      device.attributes["cpu.intel.com"].generation >= 5 &&
                      device.attributes["cpu.intel.com"].cores >= 64 &&
                      device.attributes["cpu.intel.com"].memoryGB >= 512
            # Second choice: 4th gen Xeon, relaxed core/memory requirements
            - name: xeon-gen4-medium
              deviceClassName: high-performance
              count: 1                          
              selectors:
                - cel:
                    expression: |-
                      device.attributes["cpu.intel.com"].family == "Xeon" &&
                      device.attributes["cpu.intel.com"].generation >= 4 &&
                      device.attributes["cpu.intel.com"].cores >= 32 &&
                      device.attributes["cpu.intel.com"].memoryGB >= 256

ResourceClaim == Can I have a choyxona osh with double meet and less oil

DevOps/app dev

apiVersion: resource.k8s.io/v1
kind: ResourceClaim
metadata:
  name: xeon-prioritized-claim
  namespace: default
spec:
  devices:
    requests:
      - name: cpu
        firstAvailable:
          # First choice: 5th gen Xeon, 64+ cores, AMX, high memory
          - name: xeon-gen5-large
            deviceClassName: high-performance
            count: 4                         
            selectors:
              - cel:
                  expression: |-
                    device.attributes["cpu.intel.com"].family == "Xeon" &&
                    device.attributes["cpu.intel.com"].generation >= 5 &&
                    device.attributes["cpu.intel.com"].cores >= 64 &&
                    device.attributes["cpu.intel.com"].memoryGB >= 512
          # Second choice: 4th gen Xeon, relaxed core/memory requirements
          - name: xeon-gen4-medium
            deviceClassName: high-performance
            count: 6                          
            selectors:
              - cel:
                  expression: |-
                    device.attributes["cpu.intel.com"].family == "Xeon" &&
                    device.attributes["cpu.intel.com"].generation >= 4 &&
                    device.attributes["cpu.intel.com"].cores >= 32 &&
                    device.attributes["cpu.intel.com"].memoryGB >= 256
  • creates ResourceClaims (resources that a workload needs) and attaches to the workload (Pod)

ResourceClaim == Can I have a choyxona osh with double meet and less oil

DevOps/app dev

apiVersion: resource.k8s.io/v1
kind: ResourceClaimTemplate
metadata:
  name: xeon-prioritized-claim-template
  namespace: default
spec:
  spec:
    devices:
      requests:
        - name: cpu
          firstAvailable:
            # First choice: 5th gen Xeon, 64+ cores, AMX, high memory
            - name: xeon-gen5-large
              deviceClassName: high-performance
              count: 1                          
              selectors:
                - cel:
                    expression: |-
                      device.attributes["cpu.intel.com"].family == "Xeon" &&
                      device.attributes["cpu.intel.com"].generation >= 5 &&
                      device.attributes["cpu.intel.com"].cores >= 64 &&
                      device.attributes["cpu.intel.com"].memoryGB >= 512
            # Second choice: 4th gen Xeon, relaxed core/memory requirements
            - name: xeon-gen4-medium
              deviceClassName: high-performance
              count: 1                          
              selectors:
                - cel:
                    expression: |-
                      device.attributes["cpu.intel.com"].family == "Xeon" &&
                      device.attributes["cpu.intel.com"].generation >= 4 &&
                      device.attributes["cpu.intel.com"].cores >= 32 &&
                      device.attributes["cpu.intel.com"].memoryGB >= 256
  • creates ResourceClaims (resources that a workload needs) and attaches to the workload (Pod)

ResourceClaim == Can I have a choyxona osh with double meet and less oil

ref: https://cloud.google.com/blog/products/containers-kubernetes/kubernetes-device-management-with-dra-dynamic-resource-allocation

DRA isn't only about GPU

DRA isn't only about GPU

 

  • Native resources (CPU, memory, hugepages)
  • NIC
  • RDMA net devices etc  

Let's see it in practise ...

 

Copy of Dynamic Resource Allocation with Osh.

By Feruzjon Muyassarov

Made with Slides.com

Copy of Dynamic Resource Allocation with Osh.

  • 0

More from Feruzjon Muyassarov