Controlling

docker instances

startup order

Let's talk about the order...

...the docker startup order.

For example in a docker-compose...

... prior to docker ver 1.12, order was determined by

depends_on, links, volume_from and network_mode.

However a docker instance would start up
as soon as its dependancy is up and running.

What was the problem with that?

That did not take into account the actual app state - only the fact that the docker instance is up.

I am a Solutions Engineer at

Tonight I will show you a couple of ways to deal with the docker startup order.

1. The native Docker approach

Since Docker ver 1.12 it became possible to add app level healthchecks to the docker instances

version: "2.1"
services:
  db:
  ...
    healthcheck:
      test: /usr/local/bin/healthcheck.sh

  app:
  ...
    depends_on:
      db:
        condition: service_healthy 

Demo #1

2. Consul with the ContainerPilot approach

It requires two additional components:

• Hashicorp Consul
• Joyent Triton ContainerPilot 

Hashicorp Consul - a distributed service discovery

and a key/value store

"An application-centric micro-orchestrator that automates the process of service discovery, configuration, and lifecycle management inside the container, so you can focus on your apps."

version: "3"
services:
  db:
    ...
    volumes:
      - ./containerpilot:/usr/local/bin/containerpilot
    depends_on:
      - consul

  app:
    ...
    volumes:
      - ./containerpilot:/usr/local/bin/containerpilot
    depends_on:
      - consul
      - db

  consul
    image: consul
    ports:
      - 8500:8500

ContainerPilot config file

{
  "consul": "consul:8500",
  "logging": {...},
  "preStart": "/usr/local/bin/preStart-script.sh",
  "preStop": "/usr/local/bin/preStop-script.sh",
  "postStop": "/usr/local/bin/postStop-script.sh",
  ...
  "services": [
    {
      "health": "/usr/local/bin/healthcheck.sh",
       ...
    }
  ],
  "backends": [
    {
      "onChange": "/usr/local/bin/reload-app.sh",
      ...
    }
  ],
  "tasks": [
    {
      "name": "periodic task",
      "command": "/usr/local/bin/periodic_task.sh",
      ...
    }
  ],
  "coprocesses": [
    {
      "name": "side process",
      "command": "/usr/local/bin/side-process.sh"
      ...
    }
  ]
}

• preStart - pre start command/script
• preStop - pre stop command/script
• postStop - post stop command/script
• services -> health - health check command/script
• backends -> onChange - command/script to execute if
  ip address list or port list of backend services has changed
• tasks - periodic tasks
• coprocesses - processes to run alongside with the main app
   
• telemetry - Prometheus HTTP client interface

Examples of  healthcheck script​s:

#!/bin/bash

psql -U postgres -d postgres  \
  -c "select * from information_schema.table_constraints \
  where table_name='work'" | grep '9 rows'

#!/bin/bash

[ 200 -eq $(curl -s -o /dev/null -w '%{http_code}' http://localhost:3000/works) ]

Examples of  preStart script​s:

#!/bin/bash
set -x

while :
do
  curl -s http://consul:8500/v1/health/checks/db | jq -r '.[0]["Output"]' | grep -q ok
  if [ $? == 0 ]
  then
      break
  fi
  sleep 1
done

Demo #2

https://github.com/fforloff/docker_startup_order