K8S Architecture

More Architecture Info

• Omega: flexible, scalable schedulers for large compute clusters
  • https://research.google.com/pubs/pub41684.html
• Large-scale cluster management at Google with Borg
  • https://research.google.com/pubs/pub43438.html
• Borg, Omega, and Kubernetes
  • https://research.google.com/pubs/pub44843.html
• https://github.com/kubernetes/community

Demo System

https://github.com/dalealleshouse/zero-to-devops

Setup

• Cluster
  • Minikube
• Ingress
  • Load Balancers
  • Hosts File
• Docker Registry

Deployments

• Deployments consist of pods and replica sets
  • Pod - One or more containers in a logical group
  • Replica set - controls number of pod replicas

# Create a deployment for each container in the demo system
kubectl run html-frontend --image=html-frontend:1.0 --port=80 --env STATUS_HOST=status.demo.com
kubectl run java-consumer --image=java-consumer:1.0
kubectl run ruby-producer --image=ruby-producer:1.0
kubectl run status-api --image=status-api:1.0 port=5000
kubectl run queue --image=rabbitmq:3.6.6-management

# View the pods created by the deployments
kubectl get pods

# Run docker type commands on the containers
kubectl exec -it *POD_NAME* bash
kubectl logs *POD_NAME*

Services

Services provide a durable end point

# Notice the java-consumer cannot connect to the queue
kubectl get logs *java-consumer-pod*

# The following command makes the queue discoverable via the name queue
kubectl expose deployment queue --port=15672,5672 --name=queue

# Running the command again shows that it is connected now
kubectl get logs *java-consumer-pod*

The above only creates an internal endpoint, below creates an externally accessible endpoint

# Create an endpoint for the HTML page as the REST API Service
kubectl expose deployment html-frontend --port=80 --name=html-frontend --type=NodePort
kubectl expose deployment status-api --port=80 --target-port=5000 --name=status-api --type=NodePort

The website is now externally accessible at the cluster endpoint

Infrastructure as Code

The preferred alternative to using shell commands is storing configuration in yaml files. See the kube directory

# Delete all objects made previously
# Each object has a matching file in the kube directory
kubectl delete -f kube/

# Recreate everything
kubectl create -f kube/

Default Monitoring

K8S has a default dashboard and default monitoring called heapster

# Lists cluster endpoints including the dashboard and heapster
kubectl cluster-info

# minikube only - most cloud providers have something similar
minikube addons open heapster
minikube dashboard open

Scaling

K8S will automatically load balance requests to a service between all replicas. Making subsequent requests to the html page reveals different pod names.

# Scale the NGINX deployment to 3 replicas
kubectl scale deployment html-frontend --replicas=3

K8S can create replicas easy and quickly

Auto-Scaling

K8S can scale based on load.

# Maintain between 1 and 5 replicas based on CPU usage
kubectl autoscale deployment java-consumer --min=1 --max=5 --cpu-percent=50

# Run this repeatedly to see # of replicas created
# Also, the "In Process" number on the web page will reflect the number of replicas
kubectl get deployments

Self Healing

K8S will automatically restart any pods that die.

# View all running front end docker containers
docker ps -f label=io.kubernetes.container.name=html-frontend

# Simulate a failure by forcibly removing the containers
docker rm -f *container ids*

# Containers are automatically regenerated
docker ps -f label=io.kubernetes.container.name=html-frontend

Health Checks

If the endpoint check fails, K8S automatically kills the container and starts a new one

# Find front end pod
kubectl get pods

# Simulate a failure by manually deleting the health check file
kubectl exec *POD_NAME* rm usr/share/nginx/html/healthz.html

# Notice the restart
kubectl get pods

...
        livenessProbe:
          httpGet:
            path: /healthz.html
            port: 80
          initialDelaySeconds: 3
          periodSeconds: 2
        readinessProbe:
          httpGet:
            path: /healthz.html
            port: 80
          initialDelaySeconds: 3
          periodSeconds: 2

Specify health and readiness checks in yaml

Rolling Deployment

K8S will update one pod at a time so there is no downtime for updates

# Update the image on the deployment
kubectl set image deployment/html-frontend html-frontend=html-frontend:2.0

# Run repeadly to see the number of available replicas
kubectl get deployments

Viewing the html page shows an error. K8S makes it easy to roll back deployments

# Roll back the deployment to the old image
kubectl rollout undo deployment html-frontend