Metal³
Baremetal Host Provisioning for Kubernetes
metal3.io
Who we are
Experienced Developer, @ Ericsson
Feruzjon Muyassarov
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1472361/images/7986697/ECON_RGB_48px.png)
Software Architect, @ Ericsson
Maël Kimmerlin
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1472361/images/7986697/ECON_RGB_48px.png)
Associate Manager, @
Pep Turró Mauri
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1472361/images/8013450/pasted-from-clipboard.png)
Metal³
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1472361/images/8017937/fera.jpg)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1501142/images/8030655/pep-st-benet.jpg)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1500992/images/8031480/self_bw.jpeg)
What is Metal³
- Baremetal Host Provisioning for Kubernetes
- Kubernetes native API
- An Infrastructure provider for the Cluster API (K8S SIG life-cycle)
- Self-hosted
- Self-managed
-
CNCF sandbox project
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1472361/images/7986705/pasted-from-clipboard.png)
Metal³
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1472361/images/8024596/pasted-from-clipboard.png)
Management
cluster
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1472361/images/7940854/logo.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1472361/images/8013848/pasted-from-clipboard.png)
Cluster API
What's Cluster API
Metal³
Cluster API: https://github.com/kubernetes-sigs/cluster-api
Management
cluster
Target
cluster
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1472361/images/7940854/logo.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1472361/images/7940854/logo.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1472361/images/8013848/pasted-from-clipboard.png)
Cluster API
clusterctl init \
--core cluster-api:v0.3.11 \
--bootstrap kubeadm:v0.3.11 \
--control-plane kubeadm:v0.3.11 \
--infrastructure
What's Cluster API
Metal³
Cluster API: https://github.com/kubernetes-sigs/cluster-api
Management
cluster
Target
cluster
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1472361/images/7940854/logo.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1472361/images/7940854/logo.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1472361/images/8013848/pasted-from-clipboard.png)
Cluster API
clusterctl init \
--core cluster-api:v0.3.11 \
--bootstrap kubeadm:v0.3.11 \
--control-plane kubeadm:v0.3.11 \
--infrastructure gcp
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1472361/images/8013703/pasted-from-clipboard.png)
What's Cluster API
Metal³
Cluster API: https://github.com/kubernetes-sigs/cluster-api
What's Cluster API
Management
cluster
Target
cluster
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1472361/images/7940854/logo.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1472361/images/7940854/logo.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1472361/images/8013848/pasted-from-clipboard.png)
Cluster API
clusterctl init \
--core cluster-api:v0.3.11 \
--bootstrap kubeadm:v0.3.11 \
--control-plane kubeadm:v0.3.11 \
--infrastructure aws
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1472361/images/8013703/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1472361/images/7940854/logo.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1472361/images/8013714/pasted-from-clipboard.png)
Target
cluster
Metal³
Cluster API: https://github.com/kubernetes-sigs/cluster-api
Management
cluster
Target
cluster
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1472361/images/7940854/logo.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1472361/images/7940854/logo.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1472361/images/8013848/pasted-from-clipboard.png)
Cluster API
clusterctl init \
--core cluster-api:v0.3.11 \
--bootstrap kubeadm:v0.3.11 \
--control-plane kubeadm:v0.3.11 \
--infrastructure azure
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1472361/images/8013703/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1472361/images/7940854/logo.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1472361/images/8013714/pasted-from-clipboard.png)
Target
cluster
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1472361/images/8014124/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1472361/images/7940854/logo.png)
Target
cluster
What's Cluster API
Metal³
Cluster API: https://github.com/kubernetes-sigs/cluster-api
Management
cluster
Target
cluster
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1472361/images/7940854/logo.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1472361/images/7940854/logo.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1472361/images/8013848/pasted-from-clipboard.png)
Cluster API
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1472361/images/8013703/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1472361/images/7940854/logo.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1472361/images/8013714/pasted-from-clipboard.png)
Target
cluster
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1472361/images/8014124/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1472361/images/7940854/logo.png)
Target
cluster
Target
cluster
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1472361/images/7940854/logo.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1472361/images/7885595/pasted-from-clipboard.png)
What's Cluster API
Metal³
clusterctl init \
--core cluster-api:v0.3.11 \
--bootstrap kubeadm:v0.3.11 \
--control-plane kubeadm:v0.3.11 \
--infrastructure metal3
Metal³
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1472361/images/7940854/logo.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1472361/images/7940854/logo.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1472361/images/7940854/logo.png)
Master
Worker
Worker
Metal³
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1472361/images/7940854/logo.png)
Master
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1472361/images/8018149/pasted-from-clipboard.png)
Machine
GCP
Machine
Baremetal
Operator
Metal3
Machine
AWS
Machine
BareMetalHost
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1472361/images/7885595/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1472361/images/7885595/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1472361/images/8013703/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1472361/images/8013714/pasted-from-clipboard.png)
Metal³ Stack
Metal³
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1472361/images/8014385/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1472361/images/8014392/database.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1472361/images/8014396/pasted-from-clipboard.png)
management
storage
compute
network
Metal³ Stack
Metal³
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1472361/images/8014385/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1472361/images/8014392/database.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1472361/images/8014396/pasted-from-clipboard.png)
management
storage
compute
network
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1472361/images/7940854/logo.png)
Metal³ Stack
Metal³
Ironic documentation : https://docs.openstack.org/ironic/latest/
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1472361/images/8014385/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1472361/images/8014392/database.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1472361/images/8014396/pasted-from-clipboard.png)
management
storage
compute
network
Ironic
Baremetal Operator
+
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1472361/images/7940854/logo.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1472361/images/8014428/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1472361/images/7885595/pasted-from-clipboard.png)
Metal³ Stack
Metal³
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1472361/images/8014385/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1472361/images/8014392/database.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1472361/images/8014396/pasted-from-clipboard.png)
management
storage
compute
network
Ironic documentation : https://docs.openstack.org/ironic/latest/
Ironic
Baremetal Operator
+
Cluster-api- provider-metal3
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1472361/images/7940854/logo.png)
Cluster API
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1472361/images/8013848/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1472361/images/8014428/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1472361/images/7885595/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1472361/images/7885595/pasted-from-clipboard.png)
Metal³
Metal³
Custom Controllers and Objects
Navigation...
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1472361/images/8025859/m3_github.png)
Metal³
Baremetal Operator
Metal³
1. Host Baseboard Management Controller (BMC) credentials,
BMC address
2. Host MAC address
Prerequisites
Metal³
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1472361/images/7940854/logo.png)
Baremetal
Operator
Metal³
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1472361/images/7940854/logo.png)
Secret
Baremetal
Operator
Metal³
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1472361/images/7940854/logo.png)
Secret
# Secret with BMC credentials
apiVersion: v1
kind: Secret
metadata:
name: example-host-secret
type: Opaque
data:
username: YWRtaW4= # base64
password: cGFzc3dvcmQ= # base64
Baremetal
Operator
Metal³
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1472361/images/7940854/logo.png)
Secret
Baremetal
Operator
# Secret with BMC credentials
apiVersion: v1
kind: Secret
metadata:
name: example-host-secret
type: Opaque
data:
username: YWRtaW4= # base64
password: cGFzc3dvcmQ= # base64
---
# BareMetalHost CR
apiVersion: metal3.io/v1alpha1
kind: BareMetalHost
metadata:
name: example-host
spec:
online: true
bootMACAddress: 00:34:61:e6:0d:81
bootMode: legacy
bmc:
address: ipmi://192.168.111.1:6230
credentialsName: example-host-secret
BareMetalHost
Metal³
reconcile
BareMetalHost
Baremetal
Operator
apiVersion: metal3.io/v1alpha1
kind: BareMetalHost
metadata:
finalizers:
- baremetalhost.metal3.io
generation: 1
labels:
cluster.x-k8s.io/cluster-name: test1
name: node-0
namespace: metal3
ownerReferences:
- apiVersion: infrastructure.cluster.x-k8s.io/v1alpha4
controller: true
kind: Metal3Machine
name: test1-controlplane-s6tdz
spec:
bmc:
address: ipmi://192.168.111.1:6230
credentialsName: node-0-bmc-secret
bootMACAddress: 00:8e:50:0e:e8:3a
bootMode: legacy
consumerRef:
apiVersion: infrastructure.cluster.x-k8s.io/v1alpha4
kind: Metal3Machine
name: test1-controlplane-s6tdz
namespace: metal3
image:
checksum: https://cloud.centos.org/centos/8/x86_64/images/CentOS-8.md5sum
checksumType: md5
format: qcow2
url: https://cloud.centos.org/centos/8/x86_64/images/CentOS-8-GenericCloud-8.2.2004-20200611.2.x86_64.qcow2
metaData:
name: test1-controlplane-s6tdz-metadata
namespace: metal3
networkData:
name: test1-controlplane-s6tdz-networkdata
namespace: metal3
online: true
userData:
name: test1-wxcvt
namespace: metal3
rootDeviceHints:
deviceName: /dev/sda
status:
goodCredentials:
credentials:
name: node-0-bmc-secret
namespace: metal3
credentialsVersion: "4854"
hardware:
cpu:
arch: x86_64
clockMegahertz: 2494
count: 4
flags:
- aes
...
model: Intel Xeon E3-12xx v2 (Ivy Bridge)
firmware:
bios:
...
hostname: node-0
nics:
- ip: 192.168.111.20
mac: 00:8e:50:0e:e8:3c
model: 0x1af4 0x0001
name: enp2s0
pxe: false
speedGbps: 0
vlanId: 0
- ip: 172.22.0.43
mac: 00:8e:50:0e:e8:3a
model: 0x1af4 0x0001
name: enp1s0
pxe: true
speedGbps: 0
vlanId: 0
ramMebibytes: 4096
storage:
- hctl: "0:0:0:0"
model: QEMU HARDDISK
name: /dev/sda
rotational: true
serialNumber: drive-scsi0-0-0-0
sizeBytes: 53687091200
vendor: QEMU
systemVendor:
manufacturer: QEMU
lastUpdated: "2020-10-31T02:43:10Z"
operationHistory:
deprovision:
end: null
start: null
inspect:
end: "2020-10-31T02:03:33Z"
start: "2020-10-31T01:58:50Z"
provision:
end: "2020-10-31T02:20:30Z"
start: "2020-10-31T02:15:54Z"
register:
end: "2020-10-31T02:43:10Z"
start: "2020-10-31T02:43:07Z"
operationalStatus: OK
poweredOn: true
provisioning:
ID: b84d7118-f5ca-4dea-a65a-8487c9f68d07
bootMode: legacy
image:
checksum: https://cloud.centos.org/centos/8/x86_64/images/CentOS-8.md5sum
checksumType: md5
format: qcow2
url: https://cloud.centos.org/centos/8/x86_64/images/CentOS-8-GenericCloud-8.2.2004-20200611.2.x86_64.qcow2
rootDeviceHints:
deviceName: /dev/sda
state: provisioned
triedCredentials:
credentials:
name: node-0-bmc-secret
namespace: metal3
credentialsVersion: "4854"
BareMetalHost
Metal³
Metal³
Cluster-api-provider-metal3
Cluster-api
Cluster
Controller
BareMetal
Host
Metal3
Machine
Machine
Metal3
Cluster
Cluster
Kubeadm
Config
Machine
Controller
Metal3
Cluster
Controller
Metal3
Machine
Controller
Cluster-api-
provider-metal3
Baremetal
Operator
Cluster API
Provider
Kubeadm
Object Refernce
Reconcile
apiVersion: infrastructure.cluster.x-k8s.io/v1alpha4
kind: Metal3Cluster
metadata:
name: example_cluster
spec:
controlPlaneEndpoint:
host: 192.168.111.249
port: 6443
Metal3Cluster
Metal³
apiVersion: infrastructure.cluster.x-k8s.io/v1alpha4
kind: Metal3Machine
metadata:
annotations:
metal3.io/BareMetalHost: metal3/node-0
finalizers:
- metal3machine.infrastructure.cluster.x-k8s.io
name: test1-controlplane-s6tdz
namespace: metal3
ownerReferences:
- apiVersion: cluster.x-k8s.io/v1alpha3
blockOwnerDeletion: true
controller: true
kind: Machine
name: test1-jntbq
spec:
hostSelector: {}
image:
checksum: https://cloud.centos.org/centos/8/x86_64/images/CentOS-8.md5sum
checksumType: md5
format: raw
url: https://cloud.centos.org/centos/8/x86_64/images/CentOS-8-GenericCloud-8.2.2004-20200611.2.x86_64.qcow2
providerID: metal3://fc5847cc-7cde-46f5-a4f4-99024439f6a4
status:
addresses:
- address: 192.168.111.20
type: InternalIP
- address: 172.22.0.66
type: InternalIP
- address: node-0
type: Hostname
- address: node-0
type: InternalDNS
ready: true
Metal3Machine
Metal³
Let's see a
Let's see a
Metal³
Metal³
Master
Worker
Worker
Target cluster
Metal³
bare metal servers
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1472361/images/7940854/logo.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1472361/images/7940854/logo.png)
Master
Worker
Worker
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1472361/images/7940854/logo.png)
Target cluster
Metal³
Minikube
#1
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1472361/images/7940854/logo.png)
Management cluster
Management cluster == Source cluster
Metal³
#1
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1472361/images/7940854/logo.png)
BMO
CAPM3
CAPI
BMO - Baremetal Operator
CAPM3 - Cluster-api-provider-metal3
CAPI - Cluster-api
Management cluster == Source cluster
Minikube
Management cluster
Metal³
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1472361/images/7940854/logo.png)
BMO
CAPM3
CAPI
BMO - Baremetal Operator
CAPM3 - Cluster-api-provider-metal3
CAPI - Cluster-api
Libvirt Virtual Machines
Management cluster
Minikube
#2
Metal³
Libvirt Virtual Machines
BareMetal
Host
Metal3
Machine
Metal3
Cluster
Metal3
Cluster
Cluster
Management cluster
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1472361/images/7940854/logo.png)
BMO
CAPM3
CAPI
Metal3
Machine
Minikube
#3
BMO - Baremetal Operator
CAPM3 - Cluster-api-provider-metal3
CAPI - Cluster-api
Metal³
#4
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1472361/images/7940854/logo.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1472361/images/7940854/logo.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1472361/images/7940854/logo.png)
Master
Worker
Worker
Target cluster
Management cluster
Minikube
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1472361/images/7940854/logo.png)
CAPI
CAPM3
BMO
Metal³
Interested to contribute...
- Documentations
- A new feature request
- Bug report
- Bug fixes
- Reviews
- Talks/presentations/blog posts
- Questions/feedback
We welcome you very much!
How to Contribute: https://github.com/metal3-io/cluster-api-provider-metal3/blob/master/CONTRIBUTING.md
Metal³
Metal³ Community
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1472361/images/7940915/slack.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1472361/images/7940947/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1472361/images/7942097/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1472361/images/7942100/pasted-from-clipboard.png)
Contributors: Red Hat, Ericsson, Mirantis, Dell EMC, Fujitsu, AT&T
#cluster-api-baremetal channel on K8S slack
https://groups.google.com/g/metal3-dev
Community meetings in Zoom. Every alternate Wednesday, @13:00 UTC
Github: https://github.com/metal3-io
Meeting recordings & Demos: Metal³ YouTube channel
Mailing list:
Website: https://metal3.io
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1472361/images/7885595/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1472361/images/7954659/pasted-from-clipboard.png)
@metal3_io
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1472361/images/7885627/pasted-from-clipboard.png)
- Slides: https://bit.ly/3gtBIev
- Zoom link: https://bit.ly/3mOpJev
- Community meeting recordings: https://bit.ly/3kHybKO
-
Kubernetes slack: http://slack.k8s.io/
Useful links
Metal³: Kubernetes-native bare metal host management
By fmuyassarov
Metal³: Kubernetes-native bare metal host management
Metal³ (“metal kubed”) is an open-source bare metal host provisioning tool created to enable Kubernetes-native infrastructure management. With Metal³ bare metal hosts can be managed via custom resources through the Kubernetes API. The Metal³ project is also building integration with the Kubernetes cluster-api project, allowing Metal³ to be used as an infrastructure backend for Machine objects from the Cluster API. This presentation will introduce the project and its motivations and will provide an overview of what has been accomplished so far.
- 1,012