How I met KinD

for my development environment

 

CNCF Meetup 

Turin, 28/11/2019, 

uname -a

Dario Tranchitella (prometnerion)

 

Part-time Cloud DevOps Engineer,

Full-Time dad of two.

 

I love distributed systems and whole stuff around it:
Kubernetes, Go, you know...

 

I'm mad at food.

 

Uptime

up 30 years,  90 weeks, 21 days, 18:45

1 user,  load average: 2.41, 2.62, 2.45

what's kind?

https://github.com/kubernetes-sigs/kind

 

kind is a tool for running local Kubernetes clusters using Docker container "nodes". kind is primarily designed for testing Kubernetes 1.11+, initially targeting the conformance tests.

disclaimer

ben disapproves this talk!

long

story

short

alternatives

  • minikube
  • docker compose
  • develop and test
    straightforward
    in production

minikube

PRO

  • runs in a hypervisor
  • battle-tested
  • widely adopted
  • static IP
  • load balancers
  • node ports
  • persistence

CONS

  • you need a hypervisor
  • slow file mounts
  • lots of resources
  • hard provisioning between different OSs
  • it's not declarative

minikube: provisioning

 
if [[ "$detected_OS" == "Ubuntu" ]]; then
	sudo apt-get update
	sudo apt-get -y install \
		nfs-kernel-server \
		virtualbox
	sudo systemctl enable nfs-server
fi

if [[ "$detected_OS" == "Fedora" ]]; then
	# Installing KVM Docker Machine
	curl -LO https://storage.googleapis.com/minikube/releases/latest/docker-machine-driver-kvm2
	chmod +x docker-machine-driver-kvm2
	sudo mv docker-machine-driver-kvm2 /usr/local/bin/
	# Installing system-wide dependencies
	sudo dnf update
	sudo dnf install -y \
		nfs-utils
	sudo systemctl enable rpcbind nfs-server
	sudo systemctl start rpcbind nfs-server
fi

minikube: NFS

 
sudo touch /etc/exports

if [[ "$detected_OS" == "Ubuntu" ]]; then
  SOURCES_DIR=$(dirname $(realpath .))
  EXPORT="$SOURCES_DIR $(minikube ip)(rw,subtree_check,all_squash,anonuid=$(id -u),anongid=$(id -g))"
  if ! $(grep -q "$EXPORT" /etc/exports); then
    echo "$EXPORT" | sudo tee -a /etc/exports > /dev/null && sudo systemctl restart nfs-server
  fi
  minikube ssh "sudo mkdir -p /data && sudo mount -t nfs -o nfsvers=3,tcp 192.168.99.1:$SOURCES_DIR /data"
fi

if [[ "$detected_OS" == "Fedora" ]]; then
  SOURCES_DIR=$(dirname $(realpath .))
  EXPORT="$SOURCES_DIR $(minikube ip)(rw,subtree_check,all_squash,anonuid=$(id -u),anongid=$(id -g))"
  if ! $(grep -q "$EXPORT" /etc/exports); then
    echo "$EXPORT" | sudo tee -a /etc/exports > /dev/null && sudo systemctl restart nfs-server
  fi
  # firewalld permissions
  sudo firewall-cmd --permanent --zone public --add-service mountd && \
  sudo firewall-cmd --permanent --zone public --add-service rpc-bind && \
  sudo firewall-cmd --permanent --zone public --add-service nfs && \
  sudo firewall-cmd --reload
  # SELinux flag
  sudo setsebool -P nfs_export_all_rw 1
  sudo exportfs -a
  # Minikube provisioning
  minikube ssh "sudo umount /data || true"
  minikube ssh "sudo mkdir -p /data && sudo mount -t nfs -o nfsvers=3,tcp 192.168.100.1:$SOURCES_DIR /data"
fi

compose

PRO

  • KISS
  • Docker runs anywhere
  • no provisioning needed

CONS

  • ...seriously?
  • keep aligned Helm charts and Compose
  • orchestrate services and networking
  • it's not Kubernetes!

KIND

...it's so cute!  ❤️

kind: declarative



type Cluster struct {
  TypeMeta `yaml:",inline"`
    
  Nodes []Node                             `yaml:"nodes,omitempty"`
  Networking Networking                    `yaml:"networking,omitempty"`
  KubeadmConfigPatches []string            `yaml:"kubeadmConfigPatches,omitempty"`
  KubeadmConfigPatchesJSON6902 []PatchJSON6902 `yaml:"kubeadmConfigPatchesJSON6902,omitempty"`
  ContainerdConfigPatches []string         `yaml:"containerdConfigPatches,omitempty"`
  ContainerdConfigPatchesJSON6902 []string `yaml:"containerdConfigPatchesJSON6902,omitempty"`
}


type Node struct {
  Role NodeRole                                `yaml:"role,omitempty"`
  Image string                                 `yaml:"image,omitempty"`
  ExtraMounts []Mount                          `yaml:"extraMounts,omitempty"`
  ExtraPortMappings []PortMapping              `yaml:"extraPortMappings,omitempty"`
  KubeadmConfigPatches []string                `yaml:"kubeadmConfigPatches,omitempty"`
  KubeadmConfigPatchesJSON6902 []PatchJSON6902 `yaml:"kubeadmConfigPatchesJSON6902,omitempty"`
}

kind: declarative



type PortMapping struct {
	ContainerPort int32          `yaml:"containerPort,omitempty"`
	HostPort int32               `yaml:"hostPort,omitempty"`
	ListenAddress string         `yaml:"listenAddress,omitempty"`
	Protocol PortMappingProtocol `yaml:"protocol,omitempty"`
}



type Mount struct {
	ContainerPath string         `yaml:"containerPath,omitempty"`
	HostPath string              `yaml:"hostPath,omitempty"`
	Readonly bool                `yaml:"readOnly,omitempty"`
	SelinuxRelabel bool          `yaml:"selinuxRelabel,omitempty"`
	Propagation MountPropagation `yaml:"propagation,omitempty"`
}

kind: EASY CLI

kind creates and manages local Kubernetes clusters using Docker container 'nodes'

Usage:
  kind [command]

Available Commands:
  build       Build one of [base-image, node-image]
  completion  Output shell completion code for the specified shell (bash or zsh)
  create      Creates one of [cluster]
  delete      Deletes one of [cluster]
  export      exports one of [kubeconfig, logs]
  get         Gets one of [clusters, nodes, kubeconfig]
  help        Help about any command
  load        Loads images into nodes
  version     prints the kind CLI version

Flags:
  -h, --help              help for kind
      --loglevel string   DEPRECATED: see -v instead
  -q, --quiet             silence all stderr output
  -v, --verbosity int32   info log verbosity
      --version           version for kind

Use "kind [command] --help" for more information about a command.

kind: create

Creates a local Kubernetes cluster using Docker container 'nodes'

Usage:
  kind create cluster [flags]

Flags:
      --config string       path to a kind config file
  -h, --help                help for cluster
      --image string        node docker image to use for booting the cluster
      --kubeconfig string   sets kubeconfig path instead of $KUBECONFIG or $HOME/.kube/config
      --name string         cluster context name (default "kind")
      --retain              retain nodes for debugging when cluster creation fails
      --wait duration       Wait for control plane node to be ready (default 0s)

Global Flags:
      --loglevel string   DEPRECATED: see -v instead
  -q, --quiet             silence all stderr output
  -v, --verbosity int32   info log verbosity

...IT'S SHOWTIME!

kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-plane
  extraMounts:
  - containerPath: /mnt/data
    hostPath: /path/to/your/codebase
    readOnly: false
    propagation: Bidirectional
  extraPortMappings:
  - containerPort: 80
    hostPort: 80
    listenAddress: 0.0.0.0
    protocol: TCP
  - containerPort: 443
    hostPort: 443
    listenAddress: 0.0.0.0
    protocol: TCP

how does it work...?

...jokes apart:

  • systemd
    responsible of taking care of units
  • runc
    avoiding Docker in Docker, let's use syscalls
  • kubeadm
    it's the standard for cluster bootstrapping

...but we got also cons!

cons: persistence

kubectl get storageclasses.storage.k8s.io
NAME                 PROVISIONER               AGE
standard (default)   kubernetes.io/host-path   161m
// https://github.com/kubernetes/kubernetes/blob/4c50ee993c82c6852eb3b3aa8dfa8ecc4bcfe330/pkg/volume/hostpath/host_path.go#L275-L282

// Create for hostPath simply creates a local /tmp/%/%s directory as a new PersistentVolume, default /tmp/hostpath_pv/%s.
// This Provisioner is meant for development and testing only and WILL NOT WORK in a multi-node cluster.
func (r *hostPathProvisioner) Provision(selectedNode *v1.Node, allowedTopologies []v1.TopologySelectorTerm) (*v1.PersistentVolume, error) {
	if util.CheckPersistentVolumeClaimModeBlock(r.options.PVC) {
		return nil, fmt.Errorf("%s does not support block volume provisioning", r.plugin.GetPluginName())
	}

	fullpath := fmt.Sprintf("/tmp/%s/%s", r.basePath, uuid.NewUUID())

cons: restarting

with v0.6.0 still issues while restarting

cons: kubeconfig

deleting a cluster doesn't cleanup kubeconfig

cons: docker opts

cannot customize docker commands

cons: docker images

no docker...

but kind load docker-image FTW!

cons: docs

still a wip

but #kind on Kubernetes Slack is very much alive!

...however chuck approves!

that's all folks!

How I met KinD

By Dario Tranchitella

How I met KinD

  • 1,501