What's new in Metal³

Baremetal Host Provisioning for Kubernetes

Lennart Jern @ Ericsson

10th May 2022

metal3.io

What is Metal³?

The Metal³ project (pronounced: “Metal Kubed”) provides components for bare metal host management with Kubernetes. You can enroll your bare metal machines, provision operating system images, and then, if you like, deploy Kubernetes clusters to them.

Components:

  • BareMetal Operator (BMO)
  • Cluster API Provider Meal³ (CAPM3)
  • IP Address Manager (IPAM)

Builds on and integrates with:

  • Ironic
  • Ironic Python Agent (IPA)
  • Cluster API

Metal³

Metal³

Master

Machine

GCP

Machine

Baremetal

Operator

Metal3

Machine

AWS

Machine

BareMetalHost

Metal³ Stack

Metal³

management

storage

compute

network

Metal³ Stack

Metal³

management

storage

compute

network

Metal³ Stack

Metal³

Ironic documentation : https://docs.openstack.org/ironic/latest/

management

storage

compute

network

Ironic

Baremetal Operator

+

Metal³ Stack

Metal³

management

storage

compute

network

Ironic documentation : https://docs.openstack.org/ironic/latest/

Ironic

Baremetal Operator

+

Cluster-api- provider-metal3

Cluster API

Metal³

Metal³

Custom Controllers and Objects

Cluster-api

Cluster
Controller

BareMetal

Host
 

Metal3

Machine

Machine

Metal3

Cluster

Cluster

Kubeadm

Config

Machine
Controller

Metal3
Cluster
Controller

Metal3
Machine
Controller

Cluster-api-

provider-metal3

Baremetal
Operator

Cluster API
Provider
Kubeadm

Object Refernce

Reconcile

apiVersion: infrastructure.cluster.x-k8s.io/v1alpha4
kind: Metal3Cluster
metadata:
  name: example_cluster
spec:
  controlPlaneEndpoint:
    host: 192.168.111.249
    port: 6443

Metal3Cluster

Metal³

apiVersion: infrastructure.cluster.x-k8s.io/v1alpha4
kind: Metal3Machine
metadata:
  annotations:
    metal3.io/BareMetalHost: metal3/node-0
  finalizers:
  - metal3machine.infrastructure.cluster.x-k8s.io
  name: test1-controlplane-s6tdz
  namespace: metal3
  ownerReferences:
  - apiVersion: cluster.x-k8s.io/v1alpha3
    blockOwnerDeletion: true
    controller: true
    kind: Machine
    name: test1-jntbq
spec:
  hostSelector: {}
  image:
    checksum: https://cloud.centos.org/centos/8/x86_64/images/CentOS-8.md5sum
    checksumType: md5
    format: raw
    url: https://cloud.centos.org/centos/8/x86_64/images/CentOS-8-GenericCloud-8.2.2004-20200611.2.x86_64.qcow2
  providerID: metal3://fc5847cc-7cde-46f5-a4f4-99024439f6a4
status:
  addresses:
  - address: 192.168.111.20
    type: InternalIP
  - address: 172.22.0.66
    type: InternalIP
  - address: node-0
    type: Hostname
  - address: node-0
    type: InternalDNS
  ready: true

Metal3Machine

Metal³

Managing bare metal hosts

Prerequisite: A Kubernetes cluster (the management cluster).

 

  1. Setup Ironic and BareMetal Operator
  2. Write Kubernetes manifests representing your hardware (BareMetalHosts + Secrets with credentials)
  3. Manage your hosts by making changes to the BareMetalHosts in Kubernetes. For example: set boot image, user data, clean disk and inspect hardware details.

Metal³

Bare metal Kubernetes clusters

Prerequisite: previous slide

 

  1. Setup Cluster API with the Metal³ provider
  2. Write manifests for CAPM3 resources: Metal3Cluster, IPPool, Metal3MachineTemplate, Metal3DataTemplate
  3. Write/generate manifests for CAPI resources: Cluster, KubeadmControlPlane, MachineDeployment, KubeadmConfigTemplate
  4. Manage both hosts and cluster through the Kubernetes API.

Metal³

So what's new?

  • Accepted into the CNCF Sandbox!
  • The Metal³ user-guide in similar fashion to the Cluster API book.
  • Pivot: Moving provider components and cluster API resources from one cluster to another. For example from a temporary bootstrap cluster to a target management cluster.
  • IP Address Manager: Static IP address allocation for Metal³. Similar to persistent volumes and claims.
  • Raw image streaming: Download the image directly to disk instead of first to memory, then decompress to disk.
  • End-to-end tests based on the Cluster API e2e framework.
  • CAPM3 v1beta1 and v1alpha5 API versions following CAPI v1beta1 and v1alpha4 respectively.
  • Renaming of the Metal³ provider for Cluster API: cluster-api-provider-baremetal -> cluster-api-provider-metal3.

Metal³

So what's new? - continued

  • Remediation: Cluster API feature to detect and handle unhealthy Machines. The Metal³ implementation has support for rebooting the BareMetalHosts instead of deleting and reprovisioning.
  • Disk cleaning: Choose if secondary disks should be cleaned during deprovisioning.
  • Node reuse: Avoid cleaning secondary storage and re-use the same host for example during upgrade or remediation.

Metal³

Let's see a

demo

Let's see a

Metal³

Recording: https://asciinema.org/a/493053

Metal³

Bootstrap cluster

Minikube

CAPI

CAPM3

BMO

Target cluster

Metal³

Management cluster

Minikube

CAPI

CAPM3

BMO

Management cluster

Metal³

Management cluster

Minikube

CAPI

CAPM3

BMO

Management cluster

Metal³

Interested to contribute...

  • Documentations
  • A new feature request
  • Bug report
  • Bug fixes
  • Reviews
  • Talks/presentations/blog posts
  • Questions/feedback

We welcome you very much!

How to Contribute:  https://github.com/metal3-io/cluster-api-provider-metal3/blob/master/CONTRIBUTING.md

Metal³

Metal³ Community 

Contributors:  Red Hat, Ericsson, Mirantis, Dell EMC, Fujitsu, AT&T

#cluster-api-baremetal channel on K8S slack

Community meetings in Zoom. Every Wednesday, @14:00 UTC

Github: https://github.com/metal3-io

Meeting recordings & Demos: Metal³ YouTube channel

Mailing list: https://groups.google.com/g/metal3-dev

Website: https://metal3.io

@metal3_io

Useful links

What's new in Metal³: Bare metal host provisioning for Kubernetes

By Lennart Jern

Source: fmuyassarov

What's new in Metal³: Bare metal host provisioning for Kubernetes

Metal³ (“metal kubed”) is an open-source bare metal host provisioning tool created to enable Kubernetes-native infrastructure management. With Metal³ bare metal hosts can be managed via custom resources through the Kubernetes API. The Metal³ project is also building integration with the Kubernetes cluster-api project, allowing Metal³ to be used as an infrastructure backend for Machine objects from the Cluster API.

  • 81

More from Lennart Jern