Dynamic Resource Allocaton with Osh

Feruzjon Muyassarov
Software Engineer @Ericsson Software Technology

Dynamic Resource Allocation (DRA)

  • DRA introduced a new way to request, allocate, and share accelerators (GPUs, FPGAs, NICs) across Pods declaratively.
  • The Device Plugin model only counted devices. DRA lets workloads describe what they need, not just how many.
K8S VersionDRA Status
v1.26Alpha
v1.31Major redesign
v1.32Beta
v1.34GA
v1.36New features

But before DRA
Let's order some osh...

  • You order osh, and you get a fixed combination: rice, meat, carrots, oil, spices.
  • Maybe you can choose portion (0.7/1), but not fine-grained preferences.
  • You order osh, and you get a fixed combination: rice, meat, carrots, oil, spices.
  • Maybe you can choose portion (0.7/1), but not fine-grained preferences.

Can we improvise?

  • You order osh, and you get a fixed combination: rice, meat, carrots, oil, spices.
  • Maybe you can choose portion (0.7/1), but not fine-grained preferences.

You still order osh, but now you can specify extras:

  • Add eggs
  • Include qazi
  • More meat, less oil,  etc.

 

Apart from ordering osh, you can now select the type of the osh that matches your preference

Let's order some accelerators...

Before

resources:
  limits:
    nvidia.com/gpu: 1
  requests:
    nvidia.com/gpu: 1

Device plugins model ...

You ask for: nvidia.com/gpu: 1

You could not say:

  • with xyz capabilities
  • prepared in this way

 

Before

resources:
  limits:
    nvidia.com/gpu: 1
  requests:
    nvidia.com/gpu: 1

Device plugins model ...

You ask for: nvidia.com/gpu: 1

You could not say:

  • with xyz capabilities
  • prepared in this way

 

requests:
  - name: gpu
    deviceClassName: gpu
    selectors:
      - cel:
          expression: |
            device.model == "A100" &&
            device.memory >= quantity("40Gi")

After

DRA model...

Describe what you actually need...

You can ask for:

  • nvidia.com/gpu: 1
  • product ID: A100-SXM4-40GB 
  • memory: 40 GB

 

Driver developer

Cluster admin

Application Developer/DevOps

Roles

Driver developer

Cluster admin

Application Developer/DevOps

Roles

.. is someone who understands how a piece of hardware works, basically knows writing the software that lets to control and allocate that hardware.

Driver developer

Cluster admin

Application Developer/DevOps

Roles

.. is someone who understands how a piece of hardware works, basically knows writing the software that lets to control and allocate that hardware.

 

Decides:

  • what attributes of the hardware to expose to DRA
  • what interfaces to implement to configure the node resources on the fly

 

Driver developer

Cluster admin

App developer/DevOps

Roles

.. is someone who understands how a piece of hardware works, basically knows writing the software that lets to control and allocate that hardware.

Driver developer

Cluster admin

App developer/DevOps

Roles

.. is someone who knows the application needs and defines resource requirements for the their application

Let's play those roles.

Let's play those roles.

We’re not writing the driver today :)

  • Install a driver

 

 

 

 

Cluster admin

helm install dra-driver-cpu...
helm install dra-driver-cpu...
kubectl get resourceSlices
NAME                                         NODE                 DRIVER              POOL                 AGE
kind-control-plane-dra.cpu-9q2ls             kind-control-plane   dra.cpu             kind-control-plane   20h

Cluster admin

  • Install a driver

 

 

 

 

  • Analyze the resource(s)

 

 

 

 

  • Install a driver

 

 

 

 

kubectl get resourceSlices
NAME                                         NODE                 DRIVER              POOL                 AGE
kind-control-plane-dra.cpu-9q2ls             kind-control-plane   dra.cpu             kind-control-plane   20h

Cluster admin

helm install dra-driver-cpu...
  • Install a driver

 

 

 

 

  • Analyze the resource(s)

 

 

 

 

kubectl get resourceSlices
NAME                                         NODE                 DRIVER              POOL                 AGE
kind-control-plane-dra.cpu-9q2ls             kind-control-plane   dra.cpu             kind-control-plane   20h

Cluster admin

helm install dra-driver-cpu...

ingredients == attributes

 

 

 

 

  • GPU xyz
  • 40 Gb mem
  • 3024 cores
  • Beef 1 kg
  • Rice 2 kg
  • Carrot 1 kg
  • Analyze the resource(s)

 

 

 

 

  • Install a driver

 

 

 

 

ResourceSlices (menu)

attributes:
    cpuId: 0
    socketID: 0
    numaNodeID: 1
    smtEnabled: false
    FreqMHz: 3200
    threadCount: 8
 

attributes:
    cpuId: 1
    socketID: 0
    numaNodeID: 0
    smtEnabled: true
    FreqMHz: 2800
    threadCount: 8
 

attributes:
    cpuId: 1
    socketID: 0
    numaNodeID: 1
    smtEnabled: false
    FreqMHz: 3500
    threadCount: 8
 

 admin

reads

ResourceSlices (menu)

attributes:
    cpuId: 0
    socketID: 0
    numaNodeID: 1
    smtEnabled: false
    FreqMHz: 3200
    threadCount: 8
 

attributes:
    cpuId: 1
    socketID: 0
    numaNodeID: 0
    smtEnabled: true
    FreqMHz: 2800
    threadCount: 8
 

attributes:
    cpuId: 1
    socketID: 0
    numaNodeID: 1
    smtEnabled: false
    FreqMHz: 3500
    threadCount: 8
 

high

performance

  • server a
  • server b
  • server c
     

general
purpose

  • server g
  • server f

ARM

  • server a
  • server b
  • server f
     

Intel

  • server g
  • server h
  • server c
     

DeviceClasses (categorization)

 admin

reads

creates

high

performance

  • server a
  • server b
  • server c
     

general
purpose

  • server g
  • server f

ARM

  • server a
  • server b
  • server f
     

Intel

  • server g
  • server h
  • server c
     

DeviceClasses (categorization)


 apiVersion: resource.k8s.io/v1
 kind: DeviceClass
 metadata:
   name: high-performance
 spec:
   selectors:
     - cel:
         expression: >
           device.driver == 'intel.com' &&
           device.attributes["intel.com/family"].string == "Xeon" &&
           device.attributes["intel.com/cores"].int >= 32 &&
           device.attributes["intel.com/memoryGB"].int >= 256 &&
           device.attributes["intel.com/turboBoost"].bool == true

1. Devices are attached to the Nodes

2. Device driver is installed

3. ResourceSlices and DeviceClasses are created

Recap

...it is time for the DevOps to claim

resources

  • creates ResourceClaims (resources that a workload needs) and attaches to the workload (Pod)

DevOps/app dev

ResourceClaim == Can I have a choyxona osh with double meet and less oil

  • creates ResourceClaims (resources that a workload needs) and attaches to the workload (Pod)

DevOps/app dev

apiVersion: resource.k8s.io/v1
kind: ResourceClaim
metadata:
  name: xeon-prioritized-claim
  namespace: default
spec:
  devices:
    requests:
      - name: cpu
        firstAvailable:
          # First choice: 5th gen Xeon, 64+ cores, AMX, high memory
          - name: xeon-gen5-large
            deviceClassName: high-performance
            count: 4                         
            selectors:
              - cel:
                  expression: |-
                    device.attributes["cpu.intel.com"].family == "Xeon" &&
                    device.attributes["cpu.intel.com"].generation >= 5 &&
                    device.attributes["cpu.intel.com"].cores >= 64 &&
                    device.attributes["cpu.intel.com"].memoryGB >= 512
          # Second choice: 4th gen Xeon, relaxed core/memory requirements
          - name: xeon-gen4-medium
            deviceClassName: high-performance
            count: 6                          
            selectors:
              - cel:
                  expression: |-
                    device.attributes["cpu.intel.com"].family == "Xeon" &&
                    device.attributes["cpu.intel.com"].generation >= 4 &&
                    device.attributes["cpu.intel.com"].cores >= 32 &&
                    device.attributes["cpu.intel.com"].memoryGB >= 256

ResourceClaim == Can I have a choyxona osh with double meet and less oil

  • creates ResourceClaims (resources that a workload needs) and attaches to the workload (Pod)

DevOps/app dev

apiVersion: resource.k8s.io/v1
kind: ResourceClaimTemplate
metadata:
  name: xeon-prioritized-claim-template
  namespace: default
spec:
  spec:
    devices:
      requests:
        - name: cpu
          firstAvailable:
            # First choice: 5th gen Xeon, 64+ cores, AMX, high memory
            - name: xeon-gen5-large
              deviceClassName: high-performance
              count: 1                          
              selectors:
                - cel:
                    expression: |-
                      device.attributes["cpu.intel.com"].family == "Xeon" &&
                      device.attributes["cpu.intel.com"].generation >= 5 &&
                      device.attributes["cpu.intel.com"].cores >= 64 &&
                      device.attributes["cpu.intel.com"].memoryGB >= 512
            # Second choice: 4th gen Xeon, relaxed core/memory requirements
            - name: xeon-gen4-medium
              deviceClassName: high-performance
              count: 1                          
              selectors:
                - cel:
                    expression: |-
                      device.attributes["cpu.intel.com"].family == "Xeon" &&
                      device.attributes["cpu.intel.com"].generation >= 4 &&
                      device.attributes["cpu.intel.com"].cores >= 32 &&
                      device.attributes["cpu.intel.com"].memoryGB >= 256

ResourceClaim == Can I have a choyxona osh with double meet and less oil

DevOps/app dev

apiVersion: resource.k8s.io/v1
kind: ResourceClaim
metadata:
  name: xeon-prioritized-claim
  namespace: default
spec:
  devices:
    requests:
      - name: cpu
        firstAvailable:
          # First choice: 5th gen Xeon, 64+ cores, AMX, high memory
          - name: xeon-gen5-large
            deviceClassName: high-performance
            count: 4                         
            selectors:
              - cel:
                  expression: |-
                    device.attributes["cpu.intel.com"].family == "Xeon" &&
                    device.attributes["cpu.intel.com"].generation >= 5 &&
                    device.attributes["cpu.intel.com"].cores >= 64 &&
                    device.attributes["cpu.intel.com"].memoryGB >= 512
          # Second choice: 4th gen Xeon, relaxed core/memory requirements
          - name: xeon-gen4-medium
            deviceClassName: high-performance
            count: 6                          
            selectors:
              - cel:
                  expression: |-
                    device.attributes["cpu.intel.com"].family == "Xeon" &&
                    device.attributes["cpu.intel.com"].generation >= 4 &&
                    device.attributes["cpu.intel.com"].cores >= 32 &&
                    device.attributes["cpu.intel.com"].memoryGB >= 256
  • creates ResourceClaims (resources that a workload needs) and attaches to the workload (Pod)

ResourceClaim == Can I have a choyxona osh with double meet and less oil

DevOps/app dev

apiVersion: resource.k8s.io/v1
kind: ResourceClaimTemplate
metadata:
  name: xeon-prioritized-claim-template
  namespace: default
spec:
  spec:
    devices:
      requests:
        - name: cpu
          firstAvailable:
            # First choice: 5th gen Xeon, 64+ cores, AMX, high memory
            - name: xeon-gen5-large
              deviceClassName: high-performance
              count: 1                          
              selectors:
                - cel:
                    expression: |-
                      device.attributes["cpu.intel.com"].family == "Xeon" &&
                      device.attributes["cpu.intel.com"].generation >= 5 &&
                      device.attributes["cpu.intel.com"].cores >= 64 &&
                      device.attributes["cpu.intel.com"].memoryGB >= 512
            # Second choice: 4th gen Xeon, relaxed core/memory requirements
            - name: xeon-gen4-medium
              deviceClassName: high-performance
              count: 1                          
              selectors:
                - cel:
                    expression: |-
                      device.attributes["cpu.intel.com"].family == "Xeon" &&
                      device.attributes["cpu.intel.com"].generation >= 4 &&
                      device.attributes["cpu.intel.com"].cores >= 32 &&
                      device.attributes["cpu.intel.com"].memoryGB >= 256
  • creates ResourceClaims (resources that a workload needs) and attaches to the workload (Pod)

ResourceClaim == Can I have a choyxona osh with double meet and less oil

ref: https://cloud.google.com/blog/products/containers-kubernetes/kubernetes-device-management-with-dra-dynamic-resource-allocation

DRA isn't only about GPU

DRA isn't only about GPU

 

  • Native resources (CPU, memory, hugepages)
  • NIC
  • RDMA net devices etc

Let's see it in
practise ...

 

References:

Dynamic Resource Allocation with Osh.

By Feruzjon Muyassarov

Made with Slides.com

Dynamic Resource Allocation with Osh.

  • 40

More from Feruzjon Muyassarov