Metal³

The core component behind CCD's bare metal based deployments

Lennart Jern @ Ericsson Software Technology Finland

10th May 2022

metal3.io

What is Metal³?

The Metal³ project (pronounced: “Metal Kubed”) provides components for bare metal host management with Kubernetes. You can enroll your bare metal machines, provision operating system images, and then, if you like, deploy Kubernetes clusters to them.

Components:

BareMetal Operator (BMO)
Cluster API Provider Metal³ (CAPM3)
IP Address Manager (IPAM)

Builds on and integrates with:

Ironic
Ironic Python Agent (IPA)
Cluster API

Metal³

Custom Controllers and Objects

Cluster-api

Cluster
Controller

BareMetal

Host

Metal3

Machine

Metal3

Cluster

Kubeadm

Config

Machine
Controller

Metal3
Cluster
Controller

Metal3
Machine
Controller

Cluster-api-

provider-metal3

Baremetal
Operator

Cluster API
Provider
Kubeadm

Object Refernce

Reconcile

apiVersion: infrastructure.cluster.x-k8s.io/v1alpha4
kind: Metal3Cluster
metadata:
  name: example_cluster
spec:
  controlPlaneEndpoint:
    host: 192.168.111.249
    port: 6443

Metal3Cluster

Metal³

apiVersion: infrastructure.cluster.x-k8s.io/v1alpha4
kind: Metal3Machine
metadata:
  annotations:
    metal3.io/BareMetalHost: metal3/node-0
  finalizers:
  - metal3machine.infrastructure.cluster.x-k8s.io
  name: test1-controlplane-s6tdz
  namespace: metal3
  ownerReferences:
  - apiVersion: cluster.x-k8s.io/v1alpha3
    blockOwnerDeletion: true
    controller: true
    kind: Machine
    name: test1-jntbq
spec:
  hostSelector: {}
  image:
    checksum: https://cloud.centos.org/centos/8/x86_64/images/CentOS-8.md5sum
    checksumType: md5
    format: raw
    url: https://cloud.centos.org/centos/8/x86_64/images/CentOS-8-GenericCloud-8.2.2004-20200611.2.x86_64.qcow2
  providerID: metal3://fc5847cc-7cde-46f5-a4f4-99024439f6a4
status:
  addresses:
  - address: 192.168.111.20
    type: InternalIP
  - address: 172.22.0.66
    type: InternalIP
  - address: node-0
    type: Hostname
  - address: node-0
    type: InternalDNS
  ready: true

Metal3Machine

Metal³

Metal³ Stack

Metal³

management

storage

compute

network

Ironic documentation : https://docs.openstack.org/ironic/latest/

Ironic

Baremetal Operator

Cluster-api-provider-metal3

Cluster API

Metal³

Node

Machine

GCP

Machine

Baremetal

Operator

Metal3

Machine

AWS

Machine

BareMetalHost

Managing bare metal hosts

Prerequisite: A Kubernetes cluster (the management cluster).

Setup Ironic and BareMetal Operator
Write Kubernetes manifests representing your hardware: BareMetalHosts
Manage your hosts by making changes to the BareMetalHosts in Kubernetes. For example: set boot image, user data, clean disk and inspect hardware details.

Metal³

apiVersion: metal3.io/v1alpha1
kind: BareMetalHost
metadata:
  name: node-0
spec:
  bmc:
    address: ipmi://192.168.111.1:6230
    credentialsName: node-0-bmc-secret
  bootMACAddress: 00:30:22:36:87:8f
  bootMode: legacy
  image:
    checksum: http://172.22.0.1/images/CENTOS_9_NODE_IMAGE_K8S_v1.24.1-raw.img.md5sum
    checksumType: md5
    format: raw
    url: http://172.22.0.1/images/CENTOS_9_NODE_IMAGE_K8S_v1.24.1-raw.img
...

Bare metal Kubernetes clusters

Prerequisite: previous slide

Setup Cluster API with the Metal³ provider
Write manifests for CAPM3 resources: Metal3Cluster, IPPool, Metal3MachineTemplate, Metal3DataTemplate
Write/generate manifests for CAPI resources: Cluster, KubeadmControlPlane, MachineDeployment, KubeadmConfigTemplate
Manage both hosts and cluster through the Kubernetes API.

Metal³

Metal³ and Cluster API

Metal³

Cluster

Metal3Cluster

Kubeadm-ControlPlane

M3Machine-Template

Metal3Machine

Machine-Deployment

M3Machine-Template

Metal3Machine

Machine

BareMetalHost

Created by the user

Metal³ and Cluster API

Metal³

Machine-Deployment

M3Machine-Template

Machine

KubeadmConfig-Template

M3Data-Template

KubeadmConfig

Metal3Data

Metal3DataClaim

Metal³

Bootstrap cluster

Minikube

CAPI

CAPM3

BMO

Target cluster

Running in a Metal³ controlled cluster

Move

Metal³

~~Management~~ cluster

Minikube

CAPI

CAPM3

BMO

Management cluster

Running in a Metal³ controlled cluster

Move

Metal³

Management cluster

Minikube

CAPI

CAPM3

BMO

Management cluster

Running in a Metal³ controlled cluster

Metal³

Metal³ Community

Contributors: Red Hat, Ericsson, Mirantis, Dell EMC, Fujitsu, AT&T

#cluster-api-baremetal channel on K8S slack

Community meetings in Zoom. Every Wednesday, @14:00 UTC

Github: https://github.com/metal3-io

Meeting recordings & Demos: Metal³ YouTube channel

Mailing list: https://groups.google.com/g/metal3-dev

Website: https://metal3.io

@metal3_io

Slides: https://bit.ly/3BoLhrj
Zoom link: https://bit.ly/3mOpJev
Community meeting recordings: https://bit.ly/3kHybKO
Kubernetes slack: https://slack.k8s.io/

Useful links

The Open Source Technology behind CCD

By Lennart Jern

The Open Source Technology behind CCD

Metal³ (“metal kubed”) is an open-source bare metal host provisioning tool created to enable Kubernetes-native infrastructure management. With Metal³ bare metal hosts can be managed via custom resources through the Kubernetes API. The Metal³ project is also building integration with the Kubernetes cluster-api project, allowing Metal³ to be used as an infrastructure backend for Machine objects from the Cluster API.

Metal³

Metal³

Custom Controllers and Objects

The Open Source Technology behind CCD

More from Lennart Jern