Kubernetes

A Practical guide

What is Kubernetes

  • It's an orchestration engine (as opposed to a hypervisor)
  • Rather than you manually typing docker run, etc. it does it for you in response to:
    • container crashes
    • increased load
    • request to spawn the initial setup
  • It can do it much faster and better than you. (Try launching 300000 containers manually!)
  • Tech transition experience: 1 year of transition

Kubernetes Key Benefits

  • Service Discovery/Load Balancing
  • Storage Orchestration
  • Automate Rollout/Rollback (with zero downtime!)
  • Self-healing
  • Secret & Configuration Management
  • Horizontal Scaling

Docs

Setup

Creating a private image registry

Why? And how...

  • K8 needs to pull images from somewhere, so if you are happy to get from dockerhub then you can skip this bit
  • For local we can use the registry image with docker-compose to create a registry (guide):
#docker-compose.yml
version: "3.0"

services:
  registry:
    container_name: docker-registry
    restart: always
    image: registry:2
    ports:
      - 6000:5000
    volumes:
      - docker-registry-data:/var/lib/registry

volumes:
  docker-registry-data: {}
  1. Start it: docker-compose up -d
  2. Test it: curl -ik 127.0.0.1:6000/v2/_catalog (see tags)
    • Don't use localhost because you want to ensure ip4 address

Pushing images to registry

  • Now it's working, we need to push images to it
  • You can associate images to registries via tags
docker build -t localhost:6000/basic-node-server:0.0.1 .

docker push localhost:6000/basic-node-server:0.0.1
  • -t is 'tag'. Note that you can add multiple tags to an image
  • You can use docker tag to amend them after the fact

Using the private registry in K8

sudo kubectl create secret generic regcred --from-file=.dockerconfigjson=/Users/jamessherry/.docker/config.json --type=kubernetes.io/dockerconfigjson
  • N.B. Use ABSOLUTE paths, not ~
  • Test it: kubectl get secret regcred --output=yaml
  • Apply in files, as you'll see later with 
 imagePullSecrets:
    - name: regcred

Back to K8 itself...

  • You need either full Kubernetes
    • Which should be available on most cloud platforms
  • or various local options
    • minikube (small); kind; kubeadm (full-featured)
    • we'll use the built-in functionality of docker-desktop
    • Go into Docker Desktop > settings > kubernetes and start it. (If it sticks  and stays orange you may have to restart desktop - obvs this will kill existing containers)

Kubernetes CLI

kubectl

  • already installed with Docker Desktop
  • Save yourself time by putting an alias in your ~/.bash_profile 
alias k='kubectl'

Kubectl Comands

  1. version
  2. info about the cluster
  3. info about all services, deployments, pods, replica sets, etc.
  4. Running an image as a pod
  5. Port forwarding, so it can be seen outside the cluster (by default ips are internal cluster ips)
  6. exposing ports
  7. creating resources
  8. upserting resources

 
  cheatsheet (docs)

kubectl version

kubectl cluster-info


kubectl get all




kubectl run [container-name] 
    --image=[image-name]

kubectl port-forward [pod] [ports]






kubectl expose ...

kubectl create [resource]

kubectl apply [resource]

Monitoring K8

  • You can monitor your cluster using the command line

  • BUT you can use a GUI

  • You need to create it as a 'service' (see later)

kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/v2.6.1/aio/deploy/recommended.yaml
# backup
kubectl apply -f https://kube-dashboard-recommended.s3.eu-west-2.amazonaws.com/recommended.yml

To start it:

kubectl proxy

To create it:

Endpoint: http://localhost:8001/api/v1/namespaces/kubernetes-dashboard/services/https:kubernetes-dashboard:/proxy/

If 8001 already in use

Getting the access token

Sidenote: YAML

  • YAML stands for 'YAML Ain't Markup Language'
    • It's a programmer's joke
  • It's like JSON but with less visual clutter
  • It often holds configuration information
  • K8 makes extensive use of it
    • Rather than typing the code manually, you write a YAML file and then give it to K8
  • cheatsheet
  • There are other [more advanced] versions, like toml which are used by things like netlify

The Kubernetes Model

How Kubernetes Works

Credit: https://medium.com/@tsuyoshiushio/kubernetes-in-three-diagrams-6aba8432541c

One master node controls several worker nodes

secrets ->

Kubernetes Heirachy

  1. Container (Docker, not k8)
  2. Pod
  3. Nodes
  4. Replica Set
  5. Deployment
  6. Service (exposure of a deployment)

What is a pod?

  • human is to container as space suit is to pod
  • Pods protect and monitor containers
  • They are the smallest 'part of Kubernetes' [aka Kubernetes object]
  • They are an environment for containers
  • They help you organise your app into parts (e.g. server, db, etc. by having one per pod)
  • Pod IP, memory, volumes, etc. can then be shared/restricted across containers in that pod
  • You scale horizontally by replicating them
  • Pods can live and die but do not come back to life

What is a Node (aka pod container)?

  • They wrap a group of pods
  • They share the same 'network namespace' (ip, port)
  • They have the same loopback interface (so available on localhost)
  • They resolve port binding issues, so pod ports don't collide
  • N.B. Pods cannot span nodes

What is a Replica Set

  • A replica set allows you to define how many pods are running
  • If you stipulate n pods and one dies then the replica set will ensure that another pod will be started to take its place
  • All the resources and interconnection are taken care of for you, so the new pod has the same conditions as the old pod.
  • Can only hold one type of pod

What is a Deployment

  • A deployment is a description of your desired state
  • Once created a deployment controller attempts to achieve that state by switching out old containers/pods/replica sets for new ones, with the aim of reaching that desired state!
  • Deployments allow you to do things like rolling updates with zero down-time!

What is a StatefulSet

  • ​Like Deployments but remembers pod identities
  • Pods are created in order, not all-at-once
  • [nameOfSet]-0 will always be first pod
    • Use case: spawning & seeding databases

What is a Daemonset

  • ​DaemonSets are used to ensure that some or all of your K8S nodes run a copy of a pod, which allows you to run a daemon on every node.
  • Apply with selector label in creation yaml
  • You can use NodeAffinity to filter where to apply this rule
  • When the set is deleted, so are the pods (in all the labelled nodes)
  • Think of it like 'inserting something into all pods labelled with this affinity'

What is a Service

  • A service is an abstraction that gives a set of pods:
    • An endpoint (ip address)
    • An access policy for that endpoint
  • Basically, it handles networking to allow you to access from outside
  • Or in simpler terms, it creates a micro-service
    • One possibility there is creating a load-balancer, for example...

Sidenote: Namespaces

  • Your cluster can be sub-divided into smaller virtual clusters aka 'namespaces'
  • docs
  • The main namespace is called 'default'
  • Avoid creating namespace with prefix kube-
  • If you want to create a namespace for a section of your app:
    • kubectl create namespace demo  
  • ​To delete:
    • ​kubectl delete namespace demo  

Storage

(Likely configured by your cluster admin)

K8 Supports

  • LOTS of different storage methods, inc:
    • Volumes (just like Docker)
    • PersistentVolumes
    • PersistentVolumeClaims
    • StorageClasses

Volumes

  • emptyDir - lifetime of the pod
  • hostPath - mounts on node's filesystem
  • nfs (network file system) - remote disk
  • configMaps (see later)

What is a Persistent Volume?

  • A Persistent Volume is a cluster-wide storage unit
    • It is independent from the lifetime of pods/nodes
  • A Persistent Volume Claim is a way for entities to reference/use it

What is a Storage Class

  • A template for provisioning storage
    • e.g. It can create PV[C] for you
      • DYNAMICALLY!
  • ​​So rather than the cluster admins having to repeatedly set up storage, the class will allow the system to generate storage of its own
  • Often used with those cloud storage options e.g. AWSElasticBlockStore, GCEPersistentDisk, etc.

ConfigMaps & Secrets

ConfigMaps

  • Every app has config data
  • but with everything living/dying/moving, etc. how do you share that info?
  • In K8 config is an entity in itself!
apiVersion: v1
kind: ConfigMap
metadata:
    name: db-config
    labels:
    	app: db-config
data:
    host: localhost
    port: 27017
  • kubectl create -f <[path/]file>.yml
  • Can also be created from .env files
    • kubectl create configmap [name] --from-env-file=[path]
  • Can type straight to command line with --from-literal

Consuming ConfigMaps

apiVersion: apps/v1
spec:
...
  template:
    spec:
    ....
      containers:...
      env:
      - NAME: DBPORT
        valueFrom:
          configMapKeyRef:
            key: db-config
            name: port
  • NAME is the name the var is stored under
  • key = configmap's name
  • name is cm's data key
apiVersion: apps/v1
spec:
...
  template:
    spec:
    ....
      containers:...
      env:
        envFrom:
          configMapKeyRef:
            name: db-config
  • NAME is the name the var is stored under
  • key = configmap's name
  • name is cm's data key

Load some

Load all

Consuming ConfigMaps from a volume (live reload)

apiVersion: apps/v1
spec:
  ...
  template:
    spec:
      ....
      volumes:
      	- name: db-config-vol
        configMap:
          name: db-config
      containers:
        ....
        volumeMounts:
      	  - name: db-config-vol
          mountPath: /etc/config/
  • A volume is defined and bound to a configMap
  • The container volume mounts bind to that volume
  • Now, if you change the settings in the configMap it should echo through the system (there will be a delay 30 - 60secs)

Secrets

Creating Secrets

  • kubectl create secret generic [name] --from-literal=key=value
  • kubectl create secret generic [name] --from-file=SSH-KEY=/.ssh/id_rsa.pub
  • tls instead of generic if you're working with tls certs.
    • kubectl create secret tls ${CERT_NAME} --key ${KEY_FILE} --cert ${CERT_FILE}
  • If you create from a YAML file it is only encoded in base64!! This is not secure. Don't put secrets in your YAML files in production.

Consuming Secrets

  • kubectl get secrets [optional name] [-o yaml]
apiVersion: apps/v1
spec:
...
  template:
    spec:
    ....
      containers:...
      env:
      - NAME: DBPASSWORD
        valueFrom:
          secretKeyRef:
            key: db-config
            name: db-pass
  • key is name of the secret
  • value is sub-properties in it (just like in configMap)
apiVersion: apps/v1
spec:
...
  template:
    spec:
    ....
      volumes:
       - name: secrets
         secret-name: db-password
      containers:...
        volumeMounts:
         - name: secrets
         mountPath: /etc/db-password
         readOnly: true
  • Secrets can also be gotten from volumes

Practical Demos

Practical: Deploy a container

  • Create deployment

    • kubectl create deployment nginx --image=nginx

  • Expose Service

    • ​kubectl port-forward [podname] 8080:80

  • Testing that it worked
    • For a webapp you'd then ping the ports (8080)
    • or attach to the pod: kubectl exec -it [podname] -- /bin/bash
    • or look on the dashboard (kubectl proxy)
    • Or you can use get descriptions (same for services, deployments, etc.)
      • kubectl get pods (short detail)
      • kubectl describe pods (full detail)
    • ​You can see logs too: kubectl logs [--follow] <entity>
  • Limiting Container Resources

Practical: Deleting our app

  • Delete deployment (service, etc)

    • kubectl delete <entity type> <entity name>

    • e.g. kubectl delete deployment nginx

Kubernetes & YAML

YAML is for CONFIGURATION

  • If you remember at the beginning we were saying that manually doing things takes time

  • TRUE, K8 can let us batch those task and make them easier, but at scale it's still the same thing.

  • Is there a way we can write down a description of everything we want to happen in advance and then just launch K8, passing it that description?

  • Also, what format should that take?

  • The answer to all of that is YES, and YAML

  • YAML can be used to describe and configure/launch ANY K8 structure!

YAML Commands

  • kubectl create -f <something>.yaml
    • ​Creates imperitively (will error if resource is already there)
    • If you create a deployment then add --save-config if you may want to update later
  • kubectl apply -f <something>.yaml
    • --dry-run to test it
    • apply can create!!!
  • kubectl delete -f <something>.yaml
    • ​YES! You can delete using yaml​

Practical: Deploy Pod

---
apiVersion: v1
kind: Pod
metadata:
  name: basic-server
  namespace: demo
  labels:
    app: web
spec:
  containers:
    - name: front-end
      image: basic-node-server:0.0.1
      ports:
        - containerPort: 80
  imagePullSecrets:
    - name: regcred
---
  • k apply -f ./pod.yml [-n demo]
  • 80 is the external container port (but within the cluster)
  • So that is exposed to the K8 cluster BUT not to the outside world
  • We'll need to port-forward to do that kubectl port-forward [name] <external cluster port>:<internal cluster port>
    • port-forward is good for debugging
    • It sets up a proxy that exposes that port outside the cluster
  • When we've finished we'll test it with curl/browser (k get pods)
  • then delete the pod with kubectl delete -n demo pod basic-server 
---
apiVersion: v1
kind: Pod
metadata:
  name: basic-server
  namespace: demo
  labels:
    app: web
spec:
  containers:
    - name: front-end
      image: localhost:6000/basic-node-server:0.0.1
      readinessProbe:
      	httpGet:
          path: /index.html
          port: 80
        initialDelaySeconds: 5
        periodSeconds: 30
      livenessProbe:
      	httpGet:
          path: /index.html
          port: 80
        initialDelaySeconds: 5
        timeoutSeconds: 2
        periodSeconds: 30
        failureThreshold: 1
      ports:
        - containerPort: 80
  imagePullSecrets:
    - name: regcred
---
  • Pod health needs to be monitored
  • liveness: is it still alive?
  • readiness: is it ready to receive traffic?
  • initialDelaySeconds - allows to come online
  • periodSeconds - how often to ping
  • timeoutSeconds - how long to wait before trying to contact again
  • failureThreshold - how many fails before binning

Pod Probes

Sidenote: Labels & Filters

  • K8 uses labels A LOT (because things are ephemeral)
  • You can label any k8 object and reference it anywhere else
  • kubectl label [something] [name][key]=[value] <--set
    • ​kubectl label pods my-pod env=development
  • kubectl get pods -l env=development <-- get/filter
    • ​-l is short for --selector
    • Can be used in any command but will apply to all labels
      • kubectl delete pods -l 'env in (production, development)'

Practical: Deploy ReplicaSet

apiVersion: apps/v1
kind: ReplicaSet
metadata:
  name: basic-server
  namespace: demo
  labels:
    app: web
spec:
  # modify replicas according to your case
  replicas: 3
  selector:
    matchLabels:
      tier: frontend
  template:
    metadata:
      labels:
        tier: frontend
    spec:
      containers:
        - name: front-end
          image: localhost:6000/basic-node-server:0.0.1
          ports:
            - containerPort: 80
      imagePullSecrets:
        - name: regcred
  • Same rules would apply for port-forwarding
  • Difficult with 3 though, no?! (that's why services!)
  • Go to your Docker desktop and bin one of the containers
    • You'll see a new one start right away
  • kubectl delete -n demo replicaset basic-server

Practical: Deploy Deployment

apiVersion: apps/v1
kind: Deployment
metadata:
  name: basic-server
  namespace: demo
  labels:
    app: web
spec:
  replicas: 5
  strategy:
    type: Recreate # choose a strategy
  selector:
    matchLabels:
      app: web
  template:
    metadata:
      labels:
        app: web
    spec:
      containers:
        - name: front-end
          image: localhost:6000/basic-node-server:0.0.1
          ports:
            - containerPort: 80
      imagePullSecrets:
        - name: regcred
  • To see the rollout happen: kubectl rollout status deployment.v1.apps/basic-server -n demo
  • Now, let's perform a rolling update
  • Change the image to localhost:6000/basic-node-server:0.0.2 and run the apply again

Roll out strategies

Rolling back

  • Undoing (article)
    • kubectl rollout undo deployment.v1.apps/basic-server
    • By default Kubernetes stores the last 10 ReplicaSets
      • You can change in yaml with spec.revisionHistoryLimit
  • Rolling back to a version
    • See history: kubectl rollout history deployment.v1.apps/basic-server -n demo
    • kubectl rollout undo deployment.v1.apps/basic-server --to-revision=2

Scaling

  • Scaling: kubectl scale deployment.v1.apps/basic-server -n demo --replicas=10
  • auto-scaling: kubectl autoscale deployment.v1.apps/basic-server -n demo --min=10 --max=15 --cpu-percent=80

Getting Logs

  • brew install stern (windows)
  • stern -n<namespace> <deployment> -t --since 50m
  • stern -n demo basic-server -t --since 50m
  • guide

Practical: Deploy Service

apiVersion: v1
kind: Service
metadata:
  name: basic-server-service
  namespace: demo
spec:
  type: NodePort
  selector:
    app: web
  ports:
    # By default and for convenience, the `targetPort` is set to the same value as the `port` field.
    - port: 80
      targetPort: 80
      # Optional field
      # By default and for convenience, the Kubernetes control plane will allocate a port from a range (default: 30000-32767)
      nodePort: 30007
  • apply the yaml above
  • There are various types: NodePort, LoadBalancer, External
  • NodePort is the basic

Deploying Remotely

Demo: Google Cloud

  • It's REALLY EASY
  • BEWARE of cost!
  • Create a gcloud account
  • Go to Kubernetes tab and launch a cluster
    • will take some time
    • You'll need the gcloud cli
gcloud components install gke-gcloud-auth-plugin # allow logins from CLI
gcloud auth login # Log in via browser
gcloud projects list # look for your project
gcloud config set project kube-demo-310004 #set it as current project

#do this to set a 'context' in docker desktop
# autopilot-cluster-1 is the name of the cluster, europe-west1 the zone it's in

gcloud container clusters get-credentials autopilot-cluster-1 --zone europe-west1
  • Switch to that context (see next slide)
  • k apply as you did locally...

Sidenote: Helm

  • Like npm for K8
  • docs
  • brew install helm (windows)
  • article
    • Basically you build/get a 'chart' showing what you need and what goes where and it'll auto-create

Sidenote: Jobs & CronJobs

  • Let's you run tasks on your cluster
  • video

Training