07/2015

Michaël Garrez

What is Docker ?

Wikipedia Definition

Docker is an open-source project that automates the deployment of applications inside software containers, by providing an additional layer of abstraction and automation of operating-system-level virtualization on Linux.

Linux Containers

Docker is based on LXC (Linux Containers) which allows

it to isolate containers from each other.

LXC use mainly two Linux Kernel features to acheave it :

Namespaces (Isolation of resources)
Cgroups (Isolation of resource usage as CPU/RAM)

Docker World

Docker can be split in multiple different parts :

Docker Engine
Docker Hub
Docker Company

How to use Docker ?

Docker can be used natively on Linux (and only Linux) by the simple bash command :


wget -qO- https://get.docker.com/ | sh

Docker installation requires at least 3.13 Linux Kernel.

How to use Docker ?

Docker uses boot2docker to create a VM with a Tiny Core Linux version including Docker.

How to use Docker ?

Docker now offers Docker Toolbox which contains

Docker-machine (which still uses boot2docker).

It allows to create multiple docker machines (multiple VMs).

Container VS VM

Container VS Image

Image

An image is an inert snapshot of a container.

When runned (with a run command) it becomes a container.

As images can be quite heavy they are designed to be composed of layers of other images which allows to minimize an image weight.

Images are stored on Docker Hub.

Container VS Image

Image

An image is built from a Dockerfile. A file describing how the image is supposed to behave, what it extends from, ...

docker images command lists all local images :

Container VS Image

Container

Programmatically speaking, if an image what a class, then a container would be an instance of this class.

You may launch multiple containers for the same image.

docker ps command lists all running containers :

Build our first container

$ docker run -itd ubuntu:14.04

To create a container from the official ubuntu image :

To enter into a container with your terminal :

$ docker exec -it CONTAINERID bash

To check your running containers and their ids :

$ docker ps

Turn our container in an image

As said before :

An image is a snapshot of a container

Let's take a snapshot of our container :

$ docker commit -m "First commit" CONTAINERID babacooll/myfirstimage

Build a container

You can now create as many container from this image as you want :

$ docker run -dit babacooll/myfirstimage

-d : Daemonized

-it : Allocate TTY

Image name

Dockerfile

This approach is nice but as developers

we highly prefer working in files than terminals.

That's why Docker uses Dockerfiles.

Dockerfile

A Dockerfile translation of our current image would be :

FROM ubuntu
MAINTAINER Michael Garrez <michael.garrez@gmail.com>

ENV REFRESHED_AT 2015-05-11

RUN mkdir -p /var/www

To build an image from this Dockerfile :

$ docker build -t babacooll/mysecondimage .

Container

To create a container from our new image :

$ docker run -dit babacooll/mysecondimage

-d : Daemonized

-it : Allocate TTY

Image name

Image : Apache + PHP7

Dockerfile

# use the latest Ubuntu base image
# adapted from https://registry.hub.docker.com/u/coderstephen/php7 for additional needs
FROM ubuntu:14.04
MAINTAINER Michael Garrez <michael.garrez@gmail.com>

# install packages for Apache and for compiling PHP
RUN apt-get update && apt-get install -y \
    apache2-mpm-prefork \
    apache2-prefork-dev \
    aufs-tools \
    automake \
    bison \
    btrfs-tools \
    build-essential \
    curl \
    git \
    libbz2-dev \
    libcurl4-openssl-dev \
    libmcrypt-dev \
    libxml2-dev \
    re2c

# get the latest PHP source from master branch
RUN git clone --depth=1 https://github.com/php/php-src.git /usr/local/src/php

# we're going to be working out of the PHP src directory for the compile steps
WORKDIR /usr/local/src/php
ENV PHP_DIR /usr/local/php

# configure the build
RUN ./buildconf && ./configure \
    --prefix=$PHP_DIR \
    --with-config-file-path=$PHP_DIR \
    --with-config-file-scan-dir=$PHP_DIR/conf.d \
    --with-apxs2=/usr/bin/apxs2 \
    --with-libdir=/lib/x86_64-linux-gnu \
    --enable-bcmath \
    --with-bz2 \
    --enable-calendar \
    --with-curl \
    --enable-exif \
    --enable-ftp \
    --with-ldap \
    --enable-mbstring \
    --enable-mbregex \
    --with-mcrypt \
    --with-mysqli=mysqlnd \
    --with-openssl \
    --enable-pcntl \
    --without-pear \
    --enable-pdo \
    --with-pdo-mysql=mysqlnd \
    --enable-sockets \
    --with-zip \
    --with-zlib

# compile and install
RUN make && make install

ENV PATH=$PATH:/usr/local/php/bin

# set up Apache environment variables
ENV APACHE_RUN_USER=www-data \
    APACHE_RUN_GROUP=www-data \
    APACHE_LOG_DIR=/var/log/apache2 \
    APACHE_LOCK_DIR=/var/lock/apache2 \
    APACHE_PID_FILE=/var/run/apache2.pid

# Remove default site
RUN rm -f sites-enabled/000-default.conf

# Enable additional configs and mods
ENV HTTPD_PREFIX /etc/apache2
RUN a2dismod mpm_event && a2enmod mpm_prefork
RUN ln -s $HTTPD_PREFIX/mods-available/expires.load $HTTPD_PREFIX/mods-enabled/expires.load \
    && ln -s $HTTPD_PREFIX/mods-available/headers.load $HTTPD_PREFIX/mods-enabled/headers.load \
    && ln -s $HTTPD_PREFIX/mods-available/rewrite.load $HTTPD_PREFIX/mods-enabled/rewrite.load

RUN usermod -u 1000 www-data
ENV TERM xterm

EXPOSE 80

# By default, simply start apache.
CMD /usr/sbin/apache2ctl -D FOREGROUND

Build the image

$ docker build -t babacooll/apachephp7 .

Image name

Dockerfile directory

Run a Container

$ docker run -dit babacooll/apachephp7

-d : Daemonized

-it : Allocate TTY

Image name

Let's check Apache

To know the IP of Docker Host :

D.M. Name

D.M. IP

Browse our container

http://192.168.99.100:80

Open ports

To open a port on the Container :

http://192.168.99.100:80

$ docker run -dit -p 80:80 babacooll/apachephp7

How to add PHP code ?

Solutions ?

Open Container and work in it ?

Open Container in FTP ?

How to add PHP code ?

Let's build a new Image babacooll/apachephp7project

extending babacooll/apachephp7

Dockerfile

FROM babacooll/apachephp7
MAINTAINER LFT <lft@quanta.lu>

ENV REFRESHED_AT 2015-05-11

COPY vhost.conf /etc/apache2/sites-enabled/

RUN mkdir -p /var/www/babacooll
VOLUME ["/var/www/babacooll"]

COPY application/ /var/www/babacooll

WORKDIR /var/www/babacooll
EXPOSE 80
CMD ["apache2ctl", "-D", "FOREGROUND"]

How to add PHP code ?

Vhost

<VirtualHost *:80>
  ServerName babacooll.dev
  DocumentRoot /var/www/babacooll
  #RewriteEngine On
  DirectoryIndex index.php

  <Directory /var/www/babacooll>
    Options FollowSymLinks
    AllowOverride All
    Require all granted
  </Directory>

  LogLevel info
  ErrorLog /var/log/apache2/myapp-error.log
  CustomLog /var/log/apache2/myapp-access.log combined

</VirtualHost>

<FilesMatch \.php$>
    SetHandler application/x-httpd-php
</FilesMatch>

How to add PHP code ?

Let's build our image :

$ docker build -t babacooll/apachephp7project .

Let's run a container from this image :

$ docker run -dit -p 80:80 babacooll/apachephp7project

How to add PHP code ?

Add babacooll.dev to local hosts

http://babacooll.dev:80

How to add PHP code ?

Our image contains now :

Our Project (./application)
An Apache vhost (vhost.conf)
Apache + PHP7 (babacooll/apachephp7)

How to add PHP code ?

Now let's change our index.php file

to display phpinfo() :

<?php

phpinfo();

How to add PHP code ?

As babacooll/apachephp7project is an inert image of Apache + PHP7 + Our project, a modification in my local files CAN'T affect this image.

As our container is a living babacooll/apachephp7project,

a modification in my local files CAN'T affect this container.

How to add PHP code ?

A solution would be to update files directly in the container but it would only update the current container, not all containers related to

babacooll/apachephp7project.

How to add PHP code ?

Another (better) solution is to mount dynamically

a volume in the container :

$ docker run \
-dit \
-p 80:80 \
-v //c/Users/Workspace/DockerPres/ApachePHP7Project/application:/var/www/babacooll \
babacooll/apachephp7project

Local folder to mount from

Container folder to mount on

Image : MariaDB

Docker Hub

Let's use the official mariadb repository

$ docker pull mariadb

MariaDB Container

To run a Container with MariaDB :

$ docker run \
--name mymariadb \
-e MYSQL_ROOT_PASSWORD=mypassword \
-e MYSQL_DATABASE=babacooll \
-d \
mariadb

MariaDB Container

That's all for the MariaDB container !

Link Containers

Let's link our application to MariaDB !

<?php

try {
    $dbh = new PDO('mysql:host=mariadbcontainer;dbname=babacooll', 'root', 'mypassword');
} catch (PDOException $e) {
    print "Erreur !: " . $e->getMessage() . "<br/>";
    die();
}

Link Containers

Our PHP container has no knowledge of the MariaDB container.

Let's fix that !

$ docker run \
-dit \
-p 80:80 \
--link mymariadb:mariadbcontainer \
-v //c/Users/Workspace/DockerPres/ApachePHP7Project/application:/var/www/babacooll \
babacooll/apachephp7project

Docker Hub

Docker Hub contains a lot of images for various purposes.

Docker Hub provides a free storage facility for your public images and a paying storage facility for your private images (GitHub style).

Push on Docker Hub

$ docker push babacooll/apachephp7project

Pull on Docker Hub

$ docker pull babacooll/apachephp7project

Automated Builds

Docker Hub allows you to link a GitHub repository to a Docker Hub repository.

Your GitHub repository should contain a Dockerfile on the root directory.

Each time you push on GitHub, Docker Hub (through webhooks) will pull your repository and build an image into your Docker Hub repository.

Orchestration

Running all containers by hand can

be really messy and complicated.

But Docker has a solution for us :

Docker Compose

Orchestration

For the moment our simple application requires us to do those commands :

$ docker run \
--name mymariadb \
-e MYSQL_ROOT_PASSWORD=mypassword \
-e MYSQL_DATABASE=babacooll \
-d \
mariadb

$ docker run \
-dit \
-p 80:80 \
--link mymariadb:mariadbcontainer \
-v //c/Users/Workspace/DockerPres/ApachePHP7Project/application:/var/www/babacooll \
babacooll/apachephp7project

Orchestration

Solutions ?

Put those commands into a README ?
Put those commands into a bash script ?

No.

The solution is Docker compose !

Orchestration

Docker compose works by putting a single yaml file in your project.

For example this is the corresponding docker-compose.yml file for our project :

web:
  image: babacooll/apachephp7project
  ports:
    - "80:80"
  volumes:
    - //c/Users/Workspace/DockerPres/ApachePHP7Project/application:/var/www/babacooll
  links:
    - mariadb:mariadbcontainer

mariadb:
    image: mariadb
    environment:
        MYSQL_ROOT_PASSWORD: mypassword
        MYSQL_DATABASE: babacooll

Orchestration

Our project can now be started by a simple command :

$ docker-compose up -d

Scalability

Docker compose allows you to multiply a container by using the scale command :

$ docker-compose scale mariadb=5

To allow links to properly function you have to stop and remove your web container and relaunch it :

$ docker-compose stop web
$ docker-compose rm web
$ docker-compose run -d web

Project organization

How do you organize a stack composed of multiple projects implementing docker?

Example

We decided to split our future application into multiple micro-services. Docker seems a good way to run those micro-services in a "iso"-prod way.

But we faced a problem: how do you connect all those micro-services together ?

Project organization

At this point (just beginning) we have 7 repositories providing 25 containers.

Links between containers in the same repository is easily defined (through a docker-compose.yml file) but what about trans-repositories links ?

Project organization

Docker-compose allows you to connect containers together. Example for our member zone project :

memberzoneweb:
  image: babacooll/apache-php7
  environment:
    SERVICE_HOST: memberzone.local
  ports:
    - "80:80"
  links:
    - memberzonemariadb
  volumes_from:
    - memberzonedatacontainer
  working_dir: /var/www/lancastersMemberZone

memberzonemariadb:
    image: mariadb:10
    environment:
        MYSQL_ROOT_PASSWORD: password
        MYSQL_DATABASE: memberzone

memberzonedatacontainer:
    image: debian:jessie
    volumes:
      - ./:/var/www/lancastersMemberZone
      - ./vhost:/etc/apache2/sites-enabled
    command: tail -f /dev/null

memberzonephpmyadmin:
    image: nazarpc/phpmyadmin
    ports:
      - "1234:80"
    links:
      - memberzonemariadb:mysql

Project organization

We decided to use the concept of data-only containers pattern to mount our data.

This patterns respects the single responsibility principle and eases the scalability of your applications.

Project organization

All our projects has the same organization :

docker-compose.yml at the root folder
vhost folder containing the vhost file for apache (if web container)
install.sh file at the root folder

Project organization

How to provide an easy way to launch our stack without having to understand all its projects ?

We decided to create a Stack Assembler PHP project which following goals :

Cloning / pulling all the projects
Creating a global docker-compose.yml file for the whole stack
Monitoring all our containers
Adding links between cross-projects containers
Providing an easy CLI access to the containers

Project organization

The Stack Assembler contains two config files :

stack-projects.yml

blinddrawer:
    ssh: git@github.com:quantafuture/lancastersBlindDrawer.git
    branch: v2.0.0
    tag: ~
event:
    ssh: git@github.com:quantafuture/lancastersEvent.git
    branch: master
    tag: ~
guard:
    ssh: git@github.com:quantafuture/lancastersGuard.git
    branch: master
    tag: ~
redis:
    ssh: git@github.com:quantafuture/lancastersRedis.git
    branch: master
    tag: ~
memberzone:
    ssh: git@github.com:quantafuture/lancastersMemberZone.git
    branch: master
    tag: ~
backoffice:
    ssh: git@github.com:quantafuture/lancastersBackOffice.git
    branch: master
    tag: ~
payment:
    ssh: git@github.com:quantafuture/lancastersPayment.git
    branch: master
    tag: ~

Contains all the projects of the stack with a specific branch or tag defined.

Project organization

The Stack Assembler contains two config files :

stack-links.yml

blinddrawerweb:
    guardweb: guard.local
    globalredis: redis.local
eventweb:
    guardweb: guard.local
    globalredis: redis.local
guardweb:
    globalredis: redis.local
memberzoneweb:
    guardweb: guard.local
    globalredis: redis.local
    paymentweb: payment.local
backofficeweb:
    memberzoneweb: memberzone.local
    globalredis: redis.local
    guardweb: guard.local
    paymentweb: payment.local
paymentweb:
    blinddrawerweb: blinddrawer.local
    guardweb: guard.local
    eventweb: event.local

Adds specific links between containers from different projects.

Project organization

The Stack Assembler uses multiple tools to interact with the Docker galaxy :

Docker REST API (through stage1/docker-php) for basic Docker utilization.
PHP CLI for Docker-compose utilization.
google/cadvisor to ease our stack monitoring
lancasters/docker-compose-yaml-builder to parse / builder docker-compose files
wetty for browser terminal emulation

Due to a bug with OpenSSL occurring on Yosemite we use also sequenceiq/docker-socat to allow http connection to a docker-machine.

Project organization

blinddrawerweb:
    image: babacooll/apache-php7
    ports:
        - '80:80'
    volumes_from:
        - blinddrawerdatacontainer
    environment:
        SERVICE_HOST: blinddrawer.local
    links:
        - blinddrawermariadb
        - 'guardweb:guard.local'
        - 'globalredis:redis.local'
    working_dir: /var/www/lancastersBlindDrawer
blinddrawermariadb:
    image: 'mariadb:10'
    environment:
        MYSQL_ROOT_PASSWORD: password
        MYSQL_DATABASE: blinddrawer
blinddrawerdatacontainer:
    image: 'debian:jessie'
    volumes:
        - '/Users/michaelgarrez/Docker/Projects/V2Stack/lancastersStackAssembler/app/../../lancastersBlindDrawer/:/var/www/lancastersBlindDrawer:rw'
        - '/Users/michaelgarrez/Docker/Projects/V2Stack/lancastersStackAssembler/app/../../lancastersBlindDrawer/vhost:/etc/apache2/sites-enabled:rw'
    command: 'tail -f /dev/null'
blinddrawerphpmyadmin:
    image: nazarpc/phpmyadmin
    ports:
        - '1234:80'
    links:
        - 'blinddrawermariadb:mysql'
eventweb:
    image: babacooll/apache-php7
    ports:
        - '81:80'
    volumes_from:
        - eventdatacontainer
    environment:
        SERVICE_HOST: event.local
    links:
        - eventbeanstalkd
        - 'guardweb:guard.local'
        - 'globalredis:redis.local'
eventbeanstalkd:
    image: schickling/beanstalkd
eventdatacontainer:
    image: 'debian:jessie'
    volumes:
        - '/Users/michaelgarrez/Docker/Projects/V2Stack/lancastersStackAssembler/app/../../lancastersEvent/:/var/www/lancastersEvent:rw'
        - '/Users/michaelgarrez/Docker/Projects/V2Stack/lancastersStackAssembler/app/../../lancastersEvent/vhost:/etc/apache2/sites-enabled:rw'
    command: 'tail -f /dev/null'
guardweb:
    image: babacooll/apache-php7
    ports:
        - '82:80'
    volumes_from:
        - guarddatacontainer
    environment:
        SERVICE_HOST: guard.local
    links:
        - guardmariadb
        - 'globalredis:redis.local'
    working_dir: /var/www/lancastersGuard
guardmariadb:
    image: 'mariadb:10'
    environment:
        MYSQL_ROOT_PASSWORD: password
        MYSQL_DATABASE: guard
guarddatacontainer:
    image: 'debian:jessie'
    volumes:
        - '/Users/michaelgarrez/Docker/Projects/V2Stack/lancastersStackAssembler/app/../../lancastersGuard/:/var/www/lancastersGuard:rw'
        - '/Users/michaelgarrez/Docker/Projects/V2Stack/lancastersStackAssembler/app/../../lancastersGuard/vhost:/etc/apache2/sites-enabled:rw'
    command: 'tail -f /dev/null'
guardphpmyadmin:
    image: nazarpc/phpmyadmin
    ports:
        - '1235:80'
    links:
        - 'guardmariadb:mysql'
globalredis:
    image: redis
    ports:
        - '6379:6379'
    volumes_from:
        - globaldatacontainer
    command: 'redis-server /usr/local/etc/redis/redis.conf'
globaldatacontainer:
    image: 'debian:jessie'
    volumes:
        - '/Users/michaelgarrez/Docker/Projects/V2Stack/lancastersStackAssembler/app/../../lancastersRedis/conf:/usr/local/etc/redis:rw'
    command: 'tail -f /dev/null'
memberzoneweb:
    image: babacooll/apache-php7
    ports:
        - '83:80'
    volumes_from:
        - memberzonedatacontainer
    environment:
        SERVICE_HOST: memberzone.local
    links:
        - memberzonemariadb
        - 'guardweb:guard.local'
        - 'globalredis:redis.local'
        - 'paymentweb:payment.local'
    working_dir: /var/www/lancastersMemberZone
memberzonemariadb:
    image: 'mariadb:10'
    environment:
        MYSQL_ROOT_PASSWORD: password
        MYSQL_DATABASE: memberzone
memberzonedatacontainer:
    image: 'debian:jessie'
    volumes:
        - '/Users/michaelgarrez/Docker/Projects/V2Stack/lancastersStackAssembler/app/../../lancastersMemberZone/:/var/www/lancastersMemberZone:rw'
        - '/Users/michaelgarrez/Docker/Projects/V2Stack/lancastersStackAssembler/app/../../lancastersMemberZone/vhost:/etc/apache2/sites-enabled:rw'
    command: 'tail -f /dev/null'
memberzonephpmyadmin:
    image: nazarpc/phpmyadmin
    ports:
        - '1236:80'
    links:
        - 'memberzonemariadb:mysql'
backofficeweb:
    image: babacooll/apache-php7
    ports:
        - '84:80'
    volumes_from:
        - backofficedatacontainer
    environment:
        SERVICE_HOST: backoffice.local
    links:
        - backofficemariadb
        - 'memberzoneweb:memberzone.local'
        - 'globalredis:redis.local'
        - 'guardweb:guard.local'
        - 'paymentweb:payment.local'
    working_dir: /var/www/lancastersBackOffice
backofficemariadb:
    image: 'mariadb:10'
    environment:
        MYSQL_ROOT_PASSWORD: password
        MYSQL_DATABASE: backoffice
backofficedatacontainer:
    image: 'debian:jessie'
    volumes:
        - '/Users/michaelgarrez/Docker/Projects/V2Stack/lancastersStackAssembler/app/../../lancastersBackOffice/:/var/www/lancastersBackOffice:rw'
        - '/Users/michaelgarrez/Docker/Projects/V2Stack/lancastersStackAssembler/app/../../lancastersBackOffice/vhost:/etc/apache2/sites-enabled:rw'
    command: 'tail -f /dev/null'
backofficephpmyadmin:
    image: nazarpc/phpmyadmin
    ports:
        - '1237:80'
    links:
        - 'backofficemariadb:mysql'
paymentweb:
    image: babacooll/apache-php7
    ports:
        - '85:80'
    volumes_from:
        - paymentdatacontainer
    environment:
        SERVICE_HOST: payment.local
    links:
        - paymentmariadb
        - 'blinddrawerweb:blinddrawer.local'
        - 'guardweb:guard.local'
        - 'eventweb:event.local'
    working_dir: /var/www/lancastersPayment
paymentmariadb:
    image: 'mariadb:10'
    environment:
        MYSQL_ROOT_PASSWORD: password
        MYSQL_DATABASE: payment
paymentdatacontainer:
    image: 'debian:jessie'
    volumes:
        - '/Users/michaelgarrez/Docker/Projects/V2Stack/lancastersStackAssembler/app/../../lancastersPayment/:/var/www/lancastersPayment:rw'
        - '/Users/michaelgarrez/Docker/Projects/V2Stack/lancastersStackAssembler/app/../../lancastersPayment/vhost:/etc/apache2/sites-enabled:rw'
    command: 'tail -f /dev/null'
paymentphpmyadmin:
    image: nazarpc/phpmyadmin
    ports:
        - '1238:80'
    links:
        - 'paymentmariadb:mysql'

Project organization

Stack Assembler