Containerisation

Docker & Kubernetes:

A Practical guide

What's & Why's

• Originally this stemmed from the old 'works on my machine' problem
• As you move code between environments it breaks, because:
  • Different servers
    • Have different versions of software and patches
    • Attach to different externals (e.g. databases) whose drivers may be differently versioned
  • Different injected variables
    • e.g. security keys, etc.
• The problem is consistency of environment

Other reasons

• Each proprietary vendor (AWS, Rackspace, GCE, etc.) has their own methods to deploy.
  • Sometimes their methods are complex/time consuming
  • Sometimes they rope you into using their products
  • Sometimes they may not have answers to meet your requirements
  • What you actually need is their computing power and networking, but nothing else.
  • Containerisation is about taking back control
    • You use their base resources, not their systems and processes.

• Security: Being able to quickly shut down parts of an application

History

• Although it seems as if this has just exploded over the last few years, the first example of containerisation was chroot in Linux in 1979!
• Many have tried but the tech, computing power, coherency of approach and market uptake have not been there until recently
• In the modern day there are a few different options:
  • Docker [wiki] (Google)
  • Rocket or rkt (CoreOS)
  • and then a raft of others (Apache Mesos, etc.)
• Whilst Rocket may actually win out (and you can run Docker images in Rkt containers), Docker is the standard for now, so we'll do Docker

Difference between Docker & VMs

• Smaller (No Guest OS)
• Share computing resources
• Share bins/libs

Parts of Docker

The wiki page details things well, but in short:

Parts of Docker

1. Docker Daemon
2. Objects (entities)
   1. Images
   2. Containers
   3. Services (abstracted endpoint for containers)
3. Registries (Docker Hub[/Store])

Tools

1. Docker Compose - blueprints for multi-container apps
2. Docker Swarm - container orchestration

Both redundant in the presence of Kubernetes.

Setup

Docker Daemon (dockerd)

• It's what allows containers to run
• It is a persistent process that manages Docker containers
• You get access to that process using a CLI
  • For example, you say 'docker stop myContainer' on your terminal; dockerd hears this and does so.
• You need software to have a docker daemon, such as Kubernetes, Docker For Mac or Docker Machine

Doing it!

Docker Images

A Docker image is a read-only template used to build containers. Images are used to store and ship applications.

Analogy: Images are like flatpack furniture

• When you access them you get the materials (files) and the instructions for assembly (run commands)!
• They have to be built first (analogous to assembly in the IKEA factory)
• Then sent to the store (Image registry, local or docker hub)
• Then taken by the client and executed to create the product (the live container)

Docker Images: Getting them

• Images, once built are held in registries
• Registries can be:
  • public
    • dockerhub (like github for images) - is the default store.
    • docker cloud
  • private (On-prem or Private Cloud) (docs)
  • local (on your machine, managed by dockerd
• To get an image from a remote registry, you use the CLI tools

  • docker pull node:9.1

• ​To send an image:
  • ​docker push myApp:0.0.1
• ​To see the images you have locally:

  • docker images

Theory: Docker Images: Composibility

• Images can be composed (they can extend each other)
• So you start off with a base image (e.g. node 9, or openJDK)
• That image is already built and ready to go.
• If you turned it into a container, it would have node/java running inside that container.
• BUT, whilst that's fun, and it may do for products such as our old friend the h2 database, we may want to build things of our own, so we use this as a base image  and extend it with our own child image

Docker Images: Creating

• The full assembly instructions for a container are held in a file called Dockerfile (Note: No file extention)
  • ​This is the contents and assembly list for the factory workers
• There are instructions you can give inside a docker file, one is FROM (which is like include in C langs): It will let us include all the code from the parent image's Dockerfile before our own code.
• Another instruction is the CMD command.
• ONLY THE LAST CMD COMMAND IS RESPECTED. So, if we build on top of node/java/whatever, our run command will be the one to run, not the one in the node/java base image
• So we want to FROM to get the language
• Then COPY our stuff into the container (or ADD for remote pull)
• RUN any build (shell) commands (e.g. wget modules)
• Then CMD to run our program (e.g. java myApp.jar)
• (Full list of commands here)

Docker Images: Building

Important notes:

1. The app should be self-contained
   1. Docker-ising it should only require the additions of a Dockerfile and possibly a .dockerignore
   2. These can be held outside the app dir, but are mostly held in it
   3. NO CODE NEEDS TO CHANGE FOR THIS PROCESS TO WORK
      1. It is an enhancement/wrapper
2. We can create these files on the fly in CI (Jenkins), so they don't necessarily have to be stored in the app repo.

Docker Images: Building

• Once you've got a Dockerfile you need to run the CLI to build an image:
  • docker build -t myApp:0.0.1 .
  • docker build -t <name>:<version tag (optional, default: 'latest' )> <src dir path>
• You run this command IN the directory that your app is in
  • App Dirs
  • App Files
  • Dockerfile
  • .dockerignore (like .gitignore, keeps unwanted stuff, e.g. build directory, out of the image)
• When you run the command it will tail the logs
• Errors will stop the process and you may get artifacts
• If you run docker images, you should then see myApp in the list

Practical: Let's do this with a Java App

• This app echoes System.in to System.out
• Repo is here: https://github.com/jmsherry/java-io
• So the next step is to create a Dockerfile

Answer on next page

(or if you checkout the 'dockerised' branch)

Answer

FROM openjdk:7 # Base image
COPY . /usr/src/app # copy current dir into /usr/src/app INSIDE the container's file system
WORKDIR /usr/src/app # CD to that Directory

RUN apt-get update \

    && apt-get install -y --no-install-recommends maven # get maven

RUN mvn clean package # build the jar

CMD ["java", "-jar", "./target/test-artifact-0.0.1-SNAPSHOT.jar"] # run

(Click on openjdk:7 to see what happens in a base image)

Here, our CMD overrides the CMD in openjdk:7

Practical: Create a container (docker run)

• So our app image is in our local repo (run docker images to see)
• Now we need to create an container from it:
  • docker run -it java-io:0.0.1
    • -it gives an interactive terminal (for logs)
    • -d runs it detached
    • docker exec -ti <CONTAINER_ID> bash to introspect. exit to leave.
      
  • there are lots of other flags you can use
    • Port flag: -p 8080:80 <-- opens ports on the container, 8080 internal maps to 80 external
• You can see what containers are running with docker ps -a

Final Notes

• Although that's a whirlwind tour, that's basically docker
• Final notes:
  • Saving to disk: If you write to disk in a container that is a risk. If that container is destroyed, so is your data. For persistence, you need to look at VOLUMES; a docker curated area of the host hard drive

Kubernetes

What is Kubernetes

• It's an orchestration engine (as opposed to a hypervisor)
• Rather than you manually typing docker run, etc. it does it for you in response to:
  • container crashes
  • increased load
  • request to spawn the initial setup
• It can do it much faster and better than you. (Try launching 300000 containers manually!)
• Tech transition experience: 1 year of transition

Setting Up

• You need either full Kubernetes
  • Which should be available on most cloud platforms
• or minikube locally
  • bear in mind that minikube has its own dockerd, so you'll have to point at that, not the docker for mac daemon. Stop your DFM and eval $(minikube docker-env) in your terminal window

Minikube Commands

• Minikube is a VM made for local development work
• You can deploy a single-node cluster
• minikube start (in full: minikube start --vm-driver=<vm_agent: e.g. virtualbox>)
• minikube stop
• minikube delete (You'll lose EVERY SETTING)
• minikube ip (get the clusters IP address)
• minikube dashboard - GUI for the cluster
• minikube config - gets the cluster config
• minikube logs - gets cluster operating logs
• minikube docker-env - sets the env for cluster docker containers
• minikube ssh - SSH into the cluster
• ...etc, etc. minikube --help for the full list of options

Kubernetes CLI

kubectl

Kubernetes Heirachy

• Container
• Pod
• Deployment
• Replica set

• Service (exposure of a deployment)

Example Deployment

• https://kubernetes.io/docs/tutorials/stateless-application/hello-minikube/

Practical: Deploying our app

• Create deployment

  • kubectl run myApp --image=myApp:0.0.1

• Expose Service

  • ​kubectl expose deployment myApp —type=NodePort

• Testing that it worked
  • For a webapp you'd then ping the ports
  • For us we can attach to the pod
  • You can also get info on pods from the dashboard
    • minikube dashboard or kubectl proxy
  • Or you can use get descriptions
    • kubectl get pods (short detail)
    • kubectl describe pods (full detail)
    • You can do the same for services/deployments, etc.

Updating

• What we just did was deploy manually, passing args on the CLI
• You can/should deploy automatically with YAML files
• You can have a YAML file per pod/service/deployment, etc.
• To run them, we do:
  • kubectl apply -f deployment.yaml
  • ...this is how you achieve rolling deployments
• If you run them again w/ different settings, you'll update the pod/service/deployment, etc.

Example Deployent YAML

Example Service YAML

Guide to YAML API

Structure your YAML files according to https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.9/#-strong-api-overview-strong-