Abdullah Fathi

www.fotia.com.my

DOCKER SWARM

Pautan Muat Turun

Muat Turun Slaid

Panduan DevOps

Containers Everywhere = New Problem

  • How do we automate container lifecycle?
  • How can we easily scale out/in/up/down?
  • How can we ensure our containers are re-created if they fail?
  • How can we replace containers without downtime (blue/green deploy)?
  • How can we control/track where containers get started?
  • How can we create cross-node virtual networks?
  • How can we ensure only trusted servers run our containers?
  • How can we store secrets, keys, passwords and get them to the right container (and only that container)?

Docker Swarm to The Rescue

Rolling Updates
Scaling & Load Balancing
Secure by Default

DOCKER SWARM

Swarm Mode: Built in Orchestration

  • Swarm Mode is a clustering solution built inside Docker
  • Not related to Swarm "classic" for pre-1.12 versions
  • Not enable by default, new commands once enabled
    • docker swarm
    • docker node
    • docker service
    • docker stack
    • docker secret

Docker Swarm Architecture

Source: https://docs.docker.com/engine/swarm/how-swarm-mode-works/pki/

Source: https://docs.docker.com/engine/swarm/how-swarm-mode-works/nodes/

Manager Node

  • Maintaining cluster state
  • Scheduling services
  • Serving swarm mode http endpoints

Worker Node

  • Purpose is to execute containers
  • By default, all managers are also workers

Source: https://docker-docs.uclv.cu/engine/swarm/how-swarm-mode-works/services/

Source: https://docker-docs.uclv.cu/engine/swarm/how-swarm-mode-works/services/

Create n-Node Swarm

Play With Docker

Install Docker

Docker Swarm Port

Initialize Docker Swarm

# Check if swarm has been enabled or not
docker info

# Initialize docker swarm
docker swarm init --advertise-addr <ip-addr>
  • Lots of PKI and security automation
    • Root Signing Certificate created for our Swarm
    • Certificate is issued for first Manager node
    • Join tokens are created
  • Raft database created to store root CA, configs and secrets
    • Encrypted by default on disk
    • No need for another key/value system to hold orchestration/secrets
    • Replicates logs amongst Managers via mutual TLS in "control plane"

Add Node to the Cluster

# View node list
docker node ls

# Add worker node to the cluster
docker swarm join --token ...

# Promote worker node to manager
docker node update --role manager node2

# Create docker swarm join command for manager
docker swarm join-token manager

Swarm Basic Features

# Create overlay network
docker network create --driver overlay mydrupal

# Create Postgres Service
docker service create --name psql --network mydrupal -e POSTGRES_PASSWORD=mypass postgres:14

# Create Drupal Service
docker service create --name drupal --network mydrupal -p 8822:80 drupal:9

Overlay Multi-Host Networking

  • Using --driver overlay when creating network
  • For container-to-container traffic inside a single Swarm

Swarm Basic Features

# Create elasticsearch service
docker service create --name search --replicas 2 -p 9200:9200 elasticsearch:2

Routing Mesh

  • Routes ingress (incoming) packets for a Service to proper Task
  • Load balances Swarm Services across their Tasks
  • Two way:
    • Container-to-container in a Overlay network
      (uses VIP)
    • External traffic incoming to published ports (all nodes listen)

Source: https://stackoverflow.com/questions/42510944/how-is-load-balancing-done-in-docker-swarm-mode

You can use any node that is participating in the swarm, even if there is no replica of the service in question exists on that node. So you will use Node:HostPort combination. The ingress routing mesh will route the request to an active container

Source: https://stackoverflow.com/questions/42510944/how-is-load-balancing-done-in-docker-swarm-mode

The advantage of using a proxy (HAProxy) in-front of docker swarm is, swarm nodes can reside on a private network that is accessible to the proxy server, but that is not publicly accessible. This will make your cluster secure

Swarm Basic Features

  • Stacks accept Compose files as their declarative definition for services, networks, and volumes
  • We use docker stack deploy rather then docker service create
  • Stacks manages all those objects for us, including overlay network per stack. Adds stack name to start of their name
  • New deploy: key in Compose file. Can't do build:
  • Compose now ignores deploy:, Swarm ignores build:
  • docker-compose cli not needed on Swarm server

Stacks: Production Grade Compose

Source: https://github.com/dockersamples/example-voting-app/tree/main

Sample Application

Swarm Basic Features

  • Easiest "secure" solution for storing secrets in Swarm
  • What is a Secret?
    • Usernames and passwords
    • TLS certificates and keys
    • SSH private keys
    • Any data you would prefer not be "on front page of news"
  • Centrally manage and securely transmit it to only those containers that need access to it
  • Secrets are encrypted during transit and at rest in a Docker swarm

Stacks: Production Grade Compose

Swarm Basic Features

Using Secret in Swarm Services

Source: https://docs.docker.com/engine/swarm/secrets/

Swarm Basic Features

  • Swarm Raft DB is encrypted on disk
  • Only stored on disk on Manager nodes
  • Default is Managers and Workers "control plane" is TLS + Mutual Auth
  • Secrets are first stored in Swarm, then assigned to a Service(s)
  • Only containers in assigned Service(s) can see them
  • They look like files in container but are actually in-memory fs
  • /run/secrets/ or /run/secrets/

Stacks: Production Grade Compose

Swarm Basic Features

Using Secret in Swarm Services (cont..)

Source: https://docs.docker.com/engine/swarm/secrets/

Swarm App Lifecycle

  • Provides rolling replacement of tasks/containers in a service
  • Limits downtime
  • Will replace containers for most changes
  • Create options will usually change, adding -add or -rm to them
  • Includes rollback and healthcheck options
  • Also has scale & rollback subcommand for quicker access
    • docker service scale web=4 and docker service rollback web
  • A stack deploy, when pre-existing, will issue service updates

Stacks: Production Grade Compose

Service updates

Swarm App Lifecycle

  • Supported in Dockerfile, Compose YAML, docker run, and Swarm Services
  • Docker engine will exec's the command in the container
    • (e.g. curl localhost)
  • It expects exit 0 (OK) or exit 1 (Error)
  • Three container states: starting, healthy, unhealthy
  • Healthcheck status shows up in
    docker container ls
  • Check last 5 healthchecks with docker container inspect

Stacks: Production Grade Compose

Docker Healthchecks

Swarm App Lifecycle

  • Docker run does nothing with healthchecks
  • Services will replace tasks if they fail healthcheck
  • Service updates wait for them before continuing

Stacks: Production Grade Compose

Docker Healthchecks (Cont...)

Swarm App Lifecycle

Healthchecks (Docker Run Example)

docker run \
 --health-cmd="curl -f localhost:9200/_cluster/health || false" \
 --health-interval=5s \
 --health-retries=3 \
 --health-timeout=2s \
 --health-start-period=15s \
 elasticsearch:2

Swarm App Lifecycle

Healthchecks (Dockerfile Example)

  • Options for healthcheck command
    • --interval=DURATION (default: 30s)
    • --timeout=DURATION (default: 30s)
    • --start-period=DURATION (default: 0s) (17.09+)
    • --retries=N (default: 3)
  • Basic command using default options
    • HEALTHCHECK curl -f http://localhost/ || false
  • Custom options with the command
    • HEALTHCHECK --timeout=2s --interval=3s --retries=3 \
    • CMD curl -f http://localhost/ || exit 1

Healthchecks (Compose / Stack File)

version: "3.9"
services:
 web:
 image: nginx
 healthcheck:
 test: ["CMD", "curl", "-f", "http://localhost"]
 interval: 1m30s
 timeout: 10s
 retries: 3
start_period: 1m

Swarm App Lifecycle

Explore/Misc

Shared storage volumes

Persistent storage glusterfs

Gitlab CI/CD + Swarm

Your feedback matters

There are no secrets to success. It is the result of preparation, hard work, and learning from failure. - Colin Powell

THANK YOU