A new model of device allocation in Kubernetes
Feruzjon Muyassarov @EST FI
Let's book a hotel
Device Plugins:
I need 1 room. The front desk just checks if a room is free and assigns you one. That could be next to the elevator, could have a broken AC, etc. You get a room, not a fit.
Device Plugins:
I need 1 room. The front desk just checks if a room is free and assigns you one. That could be next to the elevator, could have a broken AC, etc. You get a room, not a fit.
DRA:
I need 1 room. King bed, high floor, ocean view, etc. The system matches you to a room with exactly those attributes.
Device Plugins:
I need 1 room. The front desk just checks if a room is free and assigns you one. That could be next to the elevator, could have a broken AC, etc. You get a room, not a fit.
DRA:
I need 1 room. King bed, high floor, ocean view, etc. The system matches you to a room with exactly those attributes.
Device Plugins: integer, how many
DRA: how many + attributes
| K8S Version | DRA Status |
|---|---|
| v1.26 | Alpha |
| v1.31 | Major redesign |
| v1.32 | Beta |
| v1.34 | GA |
| v1.36 | New features |
Let's order some accelerators...
Before
resources:
limits:
nvidia.com/gpu: 1
requests:
nvidia.com/gpu: 1Device plugins model ...
You ask for: nvidia.com/gpu: 1
You could not say:
- with xyz capabilities
- prepared in this way
Before
resources:
limits:
nvidia.com/gpu: 1
requests:
nvidia.com/gpu: 1Device plugins model ...
You ask for: nvidia.com/gpu: 1
You could not say:
- with xyz capabilities
- prepared in this way
requests:
- name: gpu
deviceClassName: gpu
selectors:
- cel:
expression: |
device.model == "A100" &&
device.memory >= quantity("40Gi")After
DRA model...
Describe what you actually need...
You can ask for:
- nvidia.com/gpu: 1
- product ID: A100-SXM4-40GB
- memory: 40 GB
Driver developer
Cluster admin
Application Developer/DevOps
Roles
Driver developer
Cluster admin
Application Developer/DevOps
Roles
.. is someone who understands how a piece of hardware works, basically knows writing the software that lets to control and allocate that hardware.
Driver developer
Cluster admin
Application Developer/DevOps
Roles
.. is someone who understands how a piece of hardware works, basically knows writing the software that lets to control and allocate that hardware.
Decides:
- what attributes of the hardware to expose to DRA
- what interfaces to implement to configure the node resources on the fly
Driver developer
Cluster admin
App developer/DevOps
Roles
.. is someone who understands how a piece of hardware works, basically knows writing the software that lets to control and allocate that hardware.
Driver developer
Cluster admin
App developer/DevOps
Roles
.. is someone who knows the application needs and defines resource requirements for the their application
Let's play those roles.
Let's play those roles.
We’re not writing the driver today :)
- Install a driver
Cluster admin
helm install dra-driver-cpu...helm install dra-driver-cpu...kubectl get resourceSlices
NAME NODE DRIVER POOL AGE
kind-control-plane-dra.cpu-9q2ls kind-control-plane dra.cpu kind-control-plane 20hCluster admin
- Install a driver
- Analyze the resource(s)
- Install a driver
ResourceSlices (menu)
attributes:
cpuId: 0
socketID: 0
numaNodeID: 1
smtEnabled: false
FreqMHz: 3200
threadCount: 8
attributes:
cpuId: 1
socketID: 0
numaNodeID: 0
smtEnabled: true
FreqMHz: 2800
threadCount: 8
attributes:
cpuId: 1
socketID: 0
numaNodeID: 1
smtEnabled: false
FreqMHz: 3500
threadCount: 8
admin
reads
ResourceSlices (menu)
attributes:
cpuId: 0
socketID: 0
numaNodeID: 1
smtEnabled: false
FreqMHz: 3200
threadCount: 8
attributes:
cpuId: 1
socketID: 0
numaNodeID: 0
smtEnabled: true
FreqMHz: 2800
threadCount: 8
attributes:
cpuId: 1
socketID: 0
numaNodeID: 1
smtEnabled: false
FreqMHz: 3500
threadCount: 8
high
performance
- server a
- server b
- server c
general
purpose
- server g
- server f
ARM
- server a
- server b
- server f
Intel
- server g
- server h
- server c
DeviceClasses (categorization)
admin
reads
creates
high
performance
- server a
- server b
- server c
general
purpose
- server g
- server f
ARM
- server a
- server b
- server f
Intel
- server g
- server h
- server c
DeviceClasses (categorization)
apiVersion: resource.k8s.io/v1
kind: DeviceClass
metadata:
name: high-performance
spec:
selectors:
- cel:
expression: >
device.driver == 'intel.com' &&
device.attributes["intel.com/family"].string == "Xeon" &&
device.attributes["intel.com/cores"].int >= 32 &&
device.attributes["intel.com/memoryGB"].int >= 256 &&
device.attributes["intel.com/turboBoost"].bool == true
1. Devices are attached to the Nodes
2. Device driver is installed
3. ResourceSlices and DeviceClasses are created
Recap
...it is time for the DevOps to claim
resources
- creates ResourceClaims (resources that a workload needs) and attaches to the workload (Pod)
DevOps/app dev
ResourceClaim == I would like to book a room with the sea view
- creates ResourceClaims (resources that a workload needs) and attaches to the workload (Pod)
DevOps/app dev
apiVersion: resource.k8s.io/v1
kind: ResourceClaim
metadata:
name: xeon-prioritized-claim
namespace: default
spec:
devices:
requests:
- name: cpu
firstAvailable:
# First choice: 5th gen Xeon, 64+ cores, AMX, high memory
- name: xeon-gen5-large
deviceClassName: high-performance
count: 4
selectors:
- cel:
expression: |-
device.attributes["cpu.intel.com"].family == "Xeon" &&
device.attributes["cpu.intel.com"].generation >= 5 &&
device.attributes["cpu.intel.com"].cores >= 64 &&
device.attributes["cpu.intel.com"].memoryGB >= 512
# Second choice: 4th gen Xeon, relaxed core/memory requirements
- name: xeon-gen4-medium
deviceClassName: high-performance
count: 6
selectors:
- cel:
expression: |-
device.attributes["cpu.intel.com"].family == "Xeon" &&
device.attributes["cpu.intel.com"].generation >= 4 &&
device.attributes["cpu.intel.com"].cores >= 32 &&
device.attributes["cpu.intel.com"].memoryGB >= 256- creates ResourceClaims (resources that a workload needs) and attaches to the workload (Pod)
DevOps/app dev
apiVersion: resource.k8s.io/v1
kind: ResourceClaimTemplate
metadata:
name: xeon-prioritized-claim-template
namespace: default
spec:
spec:
devices:
requests:
- name: cpu
firstAvailable:
# First choice: 5th gen Xeon, 64+ cores, AMX, high memory
- name: xeon-gen5-large
deviceClassName: high-performance
count: 1
selectors:
- cel:
expression: |-
device.attributes["cpu.intel.com"].family == "Xeon" &&
device.attributes["cpu.intel.com"].generation >= 5 &&
device.attributes["cpu.intel.com"].cores >= 64 &&
device.attributes["cpu.intel.com"].memoryGB >= 512
# Second choice: 4th gen Xeon, relaxed core/memory requirements
- name: xeon-gen4-medium
deviceClassName: high-performance
count: 1
selectors:
- cel:
expression: |-
device.attributes["cpu.intel.com"].family == "Xeon" &&
device.attributes["cpu.intel.com"].generation >= 4 &&
device.attributes["cpu.intel.com"].cores >= 32 &&
device.attributes["cpu.intel.com"].memoryGB >= 256ResourceClaim == Can I have a choyxona osh with double meet and less oil
DevOps/app dev
apiVersion: resource.k8s.io/v1
kind: ResourceClaim
metadata:
name: xeon-prioritized-claim
namespace: default
spec:
devices:
requests:
- name: cpu
firstAvailable:
# First choice: 5th gen Xeon, 64+ cores, AMX, high memory
- name: xeon-gen5-large
deviceClassName: high-performance
count: 4
selectors:
- cel:
expression: |-
device.attributes["cpu.intel.com"].family == "Xeon" &&
device.attributes["cpu.intel.com"].generation >= 5 &&
device.attributes["cpu.intel.com"].cores >= 64 &&
device.attributes["cpu.intel.com"].memoryGB >= 512
# Second choice: 4th gen Xeon, relaxed core/memory requirements
- name: xeon-gen4-medium
deviceClassName: high-performance
count: 6
selectors:
- cel:
expression: |-
device.attributes["cpu.intel.com"].family == "Xeon" &&
device.attributes["cpu.intel.com"].generation >= 4 &&
device.attributes["cpu.intel.com"].cores >= 32 &&
device.attributes["cpu.intel.com"].memoryGB >= 256- creates ResourceClaims (resources that a workload needs) and attaches to the workload (Pod)
ResourceClaim == Can I have a choyxona osh with double meet and less oil
DevOps/app dev
apiVersion: resource.k8s.io/v1
kind: ResourceClaimTemplate
metadata:
name: xeon-prioritized-claim-template
namespace: default
spec:
spec:
devices:
requests:
- name: cpu
firstAvailable:
# First choice: 5th gen Xeon, 64+ cores, AMX, high memory
- name: xeon-gen5-large
deviceClassName: high-performance
count: 1
selectors:
- cel:
expression: |-
device.attributes["cpu.intel.com"].family == "Xeon" &&
device.attributes["cpu.intel.com"].generation >= 5 &&
device.attributes["cpu.intel.com"].cores >= 64 &&
device.attributes["cpu.intel.com"].memoryGB >= 512
# Second choice: 4th gen Xeon, relaxed core/memory requirements
- name: xeon-gen4-medium
deviceClassName: high-performance
count: 1
selectors:
- cel:
expression: |-
device.attributes["cpu.intel.com"].family == "Xeon" &&
device.attributes["cpu.intel.com"].generation >= 4 &&
device.attributes["cpu.intel.com"].cores >= 32 &&
device.attributes["cpu.intel.com"].memoryGB >= 256- creates ResourceClaims (resources that a workload needs) and attaches to the workload (Pod)
ResourceClaim == Can I have a choyxona osh with double meet and less oil
DRA isn't only about GPU
DRA isn't only about GPU
- Native resources (CPU, memory, hugepages)
- NIC
- RDMA net devices etc
Let's see it in practise ...