How I met KinD
for my development environment
CNCF Meetup
Turin, 28/11/2019,
uname -a
Dario Tranchitella (prometnerion)
Part-time Cloud DevOps Engineer,
Full-Time dad of two.
I love distributed systems and whole stuff around it:
Kubernetes, Go, you know...
I'm mad at food.
Uptime
up 30 years, 90 weeks, 21 days, 18:45
1 user, load average: 2.41, 2.62, 2.45
what's kind?
https://github.com/kubernetes-sigs/kind
kind is a tool for running local Kubernetes clusters using Docker container "nodes". kind is primarily designed for testing Kubernetes 1.11+, initially targeting the conformance tests.
disclaimer
ben disapproves this talk!
long
story
short
alternatives
- minikube
- docker compose
develop and test
straightforward
in production
minikube
PRO
- runs in a hypervisor
- battle-tested
- widely adopted
- static IP
- load balancers
- node ports
- persistence
CONS
- you need a hypervisor
- slow file mounts
- lots of resources
- hard provisioning between different OSs
- it's not declarative
minikube: provisioning
if [[ "$detected_OS" == "Ubuntu" ]]; then
sudo apt-get update
sudo apt-get -y install \
nfs-kernel-server \
virtualbox
sudo systemctl enable nfs-server
fi
if [[ "$detected_OS" == "Fedora" ]]; then
# Installing KVM Docker Machine
curl -LO https://storage.googleapis.com/minikube/releases/latest/docker-machine-driver-kvm2
chmod +x docker-machine-driver-kvm2
sudo mv docker-machine-driver-kvm2 /usr/local/bin/
# Installing system-wide dependencies
sudo dnf update
sudo dnf install -y \
nfs-utils
sudo systemctl enable rpcbind nfs-server
sudo systemctl start rpcbind nfs-server
fi
minikube: NFS
sudo touch /etc/exports
if [[ "$detected_OS" == "Ubuntu" ]]; then
SOURCES_DIR=$(dirname $(realpath .))
EXPORT="$SOURCES_DIR $(minikube ip)(rw,subtree_check,all_squash,anonuid=$(id -u),anongid=$(id -g))"
if ! $(grep -q "$EXPORT" /etc/exports); then
echo "$EXPORT" | sudo tee -a /etc/exports > /dev/null && sudo systemctl restart nfs-server
fi
minikube ssh "sudo mkdir -p /data && sudo mount -t nfs -o nfsvers=3,tcp 192.168.99.1:$SOURCES_DIR /data"
fi
if [[ "$detected_OS" == "Fedora" ]]; then
SOURCES_DIR=$(dirname $(realpath .))
EXPORT="$SOURCES_DIR $(minikube ip)(rw,subtree_check,all_squash,anonuid=$(id -u),anongid=$(id -g))"
if ! $(grep -q "$EXPORT" /etc/exports); then
echo "$EXPORT" | sudo tee -a /etc/exports > /dev/null && sudo systemctl restart nfs-server
fi
# firewalld permissions
sudo firewall-cmd --permanent --zone public --add-service mountd && \
sudo firewall-cmd --permanent --zone public --add-service rpc-bind && \
sudo firewall-cmd --permanent --zone public --add-service nfs && \
sudo firewall-cmd --reload
# SELinux flag
sudo setsebool -P nfs_export_all_rw 1
sudo exportfs -a
# Minikube provisioning
minikube ssh "sudo umount /data || true"
minikube ssh "sudo mkdir -p /data && sudo mount -t nfs -o nfsvers=3,tcp 192.168.100.1:$SOURCES_DIR /data"
fi
compose
PRO
- KISS
- Docker runs anywhere
- no provisioning needed
CONS
...seriously?- keep aligned Helm charts and Compose
- orchestrate services and networking
- it's not Kubernetes!
KIND
...it's so cute! ❤️
kind: declarative
type Cluster struct {
TypeMeta `yaml:",inline"`
Nodes []Node `yaml:"nodes,omitempty"`
Networking Networking `yaml:"networking,omitempty"`
KubeadmConfigPatches []string `yaml:"kubeadmConfigPatches,omitempty"`
KubeadmConfigPatchesJSON6902 []PatchJSON6902 `yaml:"kubeadmConfigPatchesJSON6902,omitempty"`
ContainerdConfigPatches []string `yaml:"containerdConfigPatches,omitempty"`
ContainerdConfigPatchesJSON6902 []string `yaml:"containerdConfigPatchesJSON6902,omitempty"`
}
type Node struct {
Role NodeRole `yaml:"role,omitempty"`
Image string `yaml:"image,omitempty"`
ExtraMounts []Mount `yaml:"extraMounts,omitempty"`
ExtraPortMappings []PortMapping `yaml:"extraPortMappings,omitempty"`
KubeadmConfigPatches []string `yaml:"kubeadmConfigPatches,omitempty"`
KubeadmConfigPatchesJSON6902 []PatchJSON6902 `yaml:"kubeadmConfigPatchesJSON6902,omitempty"`
}
kind: declarative
type PortMapping struct {
ContainerPort int32 `yaml:"containerPort,omitempty"`
HostPort int32 `yaml:"hostPort,omitempty"`
ListenAddress string `yaml:"listenAddress,omitempty"`
Protocol PortMappingProtocol `yaml:"protocol,omitempty"`
}
type Mount struct {
ContainerPath string `yaml:"containerPath,omitempty"`
HostPath string `yaml:"hostPath,omitempty"`
Readonly bool `yaml:"readOnly,omitempty"`
SelinuxRelabel bool `yaml:"selinuxRelabel,omitempty"`
Propagation MountPropagation `yaml:"propagation,omitempty"`
}
kind: EASY CLI
kind creates and manages local Kubernetes clusters using Docker container 'nodes'
Usage:
kind [command]
Available Commands:
build Build one of [base-image, node-image]
completion Output shell completion code for the specified shell (bash or zsh)
create Creates one of [cluster]
delete Deletes one of [cluster]
export exports one of [kubeconfig, logs]
get Gets one of [clusters, nodes, kubeconfig]
help Help about any command
load Loads images into nodes
version prints the kind CLI version
Flags:
-h, --help help for kind
--loglevel string DEPRECATED: see -v instead
-q, --quiet silence all stderr output
-v, --verbosity int32 info log verbosity
--version version for kind
Use "kind [command] --help" for more information about a command.
kind: create
Creates a local Kubernetes cluster using Docker container 'nodes'
Usage:
kind create cluster [flags]
Flags:
--config string path to a kind config file
-h, --help help for cluster
--image string node docker image to use for booting the cluster
--kubeconfig string sets kubeconfig path instead of $KUBECONFIG or $HOME/.kube/config
--name string cluster context name (default "kind")
--retain retain nodes for debugging when cluster creation fails
--wait duration Wait for control plane node to be ready (default 0s)
Global Flags:
--loglevel string DEPRECATED: see -v instead
-q, --quiet silence all stderr output
-v, --verbosity int32 info log verbosity
...IT'S SHOWTIME!
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-plane
extraMounts:
- containerPath: /mnt/data
hostPath: /path/to/your/codebase
readOnly: false
propagation: Bidirectional
extraPortMappings:
- containerPort: 80
hostPort: 80
listenAddress: 0.0.0.0
protocol: TCP
- containerPort: 443
hostPort: 443
listenAddress: 0.0.0.0
protocol: TCP
how does it work...?
...jokes apart:
- systemd
responsible of taking care of units - runc
avoiding Docker in Docker, let's use syscalls - kubeadm
it's the standard for cluster bootstrapping
...but we got also cons!
cons: persistence
kubectl get storageclasses.storage.k8s.io
NAME PROVISIONER AGE
standard (default) kubernetes.io/host-path 161m
// https://github.com/kubernetes/kubernetes/blob/4c50ee993c82c6852eb3b3aa8dfa8ecc4bcfe330/pkg/volume/hostpath/host_path.go#L275-L282
// Create for hostPath simply creates a local /tmp/%/%s directory as a new PersistentVolume, default /tmp/hostpath_pv/%s.
// This Provisioner is meant for development and testing only and WILL NOT WORK in a multi-node cluster.
func (r *hostPathProvisioner) Provision(selectedNode *v1.Node, allowedTopologies []v1.TopologySelectorTerm) (*v1.PersistentVolume, error) {
if util.CheckPersistentVolumeClaimModeBlock(r.options.PVC) {
return nil, fmt.Errorf("%s does not support block volume provisioning", r.plugin.GetPluginName())
}
fullpath := fmt.Sprintf("/tmp/%s/%s", r.basePath, uuid.NewUUID())
cons: restarting
with v0.6.0 still issues while restarting
cons: kubeconfig
deleting a cluster doesn't cleanup kubeconfig
cons: docker opts
cannot customize docker commands
cons: docker images
no docker...
but kind load docker-image FTW!
cons: docs
still a wip
but #kind on Kubernetes Slack is very much alive!
...however chuck approves!
that's all folks!
How I met KinD
By Dario Tranchitella
How I met KinD
- 1,477