Docker workshop
LEVEL DEVil
il
About me
Developer. IBMer. Vi(m) lover. DevOps kid. Performing cloud infrastructure and application architecture with passion for the edge thing.
I am not obsessed with emails and spreadsheet thing as an ordinary IBMers. Reach me on Twitter as @epcim.
Workshop agenda
Morning | LEVEL DEVil ~ about the 'event' DevOps, Docker intro Docker usage in Watson team (Martin Cochner) Learn core principles, volumes, networking, orchestration. Develop custom container to host micro-service. Consume community containers. |
Afternoon | Hack session * Leverage the skills and build something worth! * Drone.io/Jenkins CI pipeline, ... * Containers on Bluemix, ... * << bring your own ideas to work on >> |
T | Day 1 |
---|
Something about you
Background | Practice | Expectations
Buzzword
By Definition
IBM
DevOps is an enterprise capability for continuous software delivery that enables clients to seize market opportunities and reduce time to customer feedback
Pick #1
Chef.io
A cultural and professional movement, focused on how we build and operate high velocity organizations, born from the experiences of its practitioners.
Pick #2
Three ways
The principles that all of the DevOps patterns can be derived from
Gee Kim, Co-Author of: “The Phoenix Project: A Novel About IT, DevOps, and Helping Your Business Win.”
![](https://s3.amazonaws.com/media-p.slid.es/uploads/81113/images/2311075/devops1.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/81113/images/2311076/devops2.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/81113/images/2311077/devops3.png)
Pick #3
Forget DEV to OPS !
Practices
- Automation
- Continuous Delivery
- Collaboration across teams
- Extended visibility
- Amplified feedback loops, rapid feedback
Feedback loop
- Do something
- Test, Measure
- Analyze
- Correct
- Repeat
Business value
- Enhanced customer experience
- Increased capacity to innovate
- Faster time to value
Wait a while ...
What did the Waterfall, Agile, Scrum done wrong?
retired buzzwords
Design Thinking
- Learning about the audience, customer view.
- Aptly vs. real needs
- Ideal vs. realistic
DevOps
- Feedback loops, cooperation, integration
- Continuous Delivery
Agile
- #12 principles, but "frequent delivery" shines
- ...
What are agile trends?
What are agile trends?
-
From traditional IT to microservices
-
API loosely coupled architectures and micro services
-
Technology shift on frontends
-
Design for speed and simplicity
-
Cross platform requirements (Go, mobile platforms)
-
Measure for feedback, data analytics, visualization
-
Stream data
-
Dynamic configuration
-
Service discovery
-
...
Chef Style DevOps Kung Fu
- Get Slides or watch Recording
- We work together to develop our principles, forms, and applications
- Join by PR, http://github.com/chef/devops-kungfu
- Start your own school by forking, and changing the content
![Creative Commons License](https://i.creativecommons.org/l/by-sa/4.0/88x31.png)
Deploy and operate all infrastructures by code, as applications are code too.
As traditional IT becomes "truck" and cloud native is a "car". Practice DevOps and apply cloud architectures to acquire expertise, tools and operational skills.
Personal picks
- DevOps approach
- Life-cycle in mind
- Meet deadlines
- Independent dependability
- One artifact for all env.
- Same deployment process
- Four eyes rule; visibility
- Team is not isolated unit
- Share
![](https://s3.amazonaws.com/media-p.slid.es/uploads/81113/images/1798494/chef-southPark.png)
Tools
Preffered languages and tools that fit the job
- Kitchen CI
- Drone.io
- ServerSpec , InSpec
- Fog cloud libs
- Virtualbox
- Consul
- ELK stack
- Grafana
- Kubernetes
- Bluemix
Backup slides
Vagrant
Kibana (ELK stack)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/81113/images/2311623/Kibana.png)
Grafana
![](https://s3.amazonaws.com/media-p.slid.es/uploads/81113/images/2311620/grafana2.png)
Consul.io
![](https://s3.amazonaws.com/media-p.slid.es/uploads/81113/images/2324036/consul.png)
Consul.io
![](https://s3.amazonaws.com/media-p.slid.es/uploads/81113/images/2311626/consul-arch-b2478674.png)
Break
(~ 5 min)
Docker
Docker
- Docker Engine
- Docker Hub
- Docker Machine
- Docker Compose
- Docker Toolbox
- Docker Swarm
- Docker Registry
- . . .
Docker
Client
Daemon
HTTP :2376
Docker
Filesystem
![](https://s3.amazonaws.com/media-p.slid.es/uploads/81113/images/2311631/docker-layers.png)
Docker
VM vs Container
![](https://s3.amazonaws.com/media-p.slid.es/uploads/81113/images/2315254/what-is-docker-diagram.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/81113/images/2315253/what-is-vm-diagram.png)
Docker
Architecture
Docker
# Start daemon manually
$ docker daemon -d
$ docker ps
# Expose external host
$ docker -H tcp://192.168.10.1:2376 ps
$ export DOCKER_HOST="tcp://192.168.10.1:2376"
$ docker ps
# Win/OSX - Start docker-machine
$ docker-machine create --driver virtualbox demo
$ docker-machine ls
NAME ACTIVE DRIVER STATE URL SWARM
demo - virtualbox Running tcp://192.168.99.100:2376
$ docker-machine env demo
export DOCKER_TLS_VERIFY="1"
export DOCKER_HOST="tcp://192.168.99.100:2376"
export DOCKER_CERT_PATH="$HOME/.docker/machine/machines/demo"
export DOCKER_MACHINE_NAME="demo"
# Run this command to configure your shell:
eval "$(/usr/local/bin/docker-machine env demo)"
Docker CLI
Usage: docker [OPTIONS] COMMAND [arg...]
docker daemon [ --help | ... ]
docker [ --help | -v | --version ]
A self-sufficient runtime for containers.
Commands:
attach Attach to a running container
build Build an image from a Dockerfile
commit Create a new image from a container's changes
cp Copy files/folders between a container and the local filesystem
create Create a new container
diff Inspect changes on a container's filesystem
events Get real time events from the server
exec Run a command in a running container
export Export a container's filesystem as a tar archive
history Show the history of an image
images List images
import Import the contents from a tarball to create a filesystem image
info Display system-wide information
inspect Return low-level information on a container or image
kill Kill a running container
load Load an image from a tar archive or STDIN
login Log in to a Docker registry
logout Log out from a Docker registry
logs Fetch the logs of a container
network Manage Docker networks
pause Pause all processes within a container
port List port mappings or a specific mapping for the CONTAINER
ps List containers
pull Pull an image or a repository from a registry
push Push an image or a repository to a registry
rename Rename a container
restart Restart a container
rm Remove one or more containers
rmi Remove one or more images
run Run a command in a new container
save Save one or more images to a tar archive
search Search the Docker Hub for images
start Start one or more stopped containers
stats Display a live stream of container(s) resource usage statistics
stop Stop a running container
tag Tag an image into a repository
top Display the running processes of a container
unpause Unpause all processes within a container
update Update configuration of one or more containers
version Show the Docker version information
volume Manage Docker volumes
wait Block until a container stops, then print its exit code
Docker machine
$ tree ~/.docker/
/home/userdir/.docker/
└── machine
├── certs
│ ├── ca-key.pem
│ ├── ca.pem
│ ├── cert.pem
│ └── key.pem
└── machines
├── demo
│ ├── boot2docker.iso
│ ├── ca.pem
│ ├── cert.pem
│ ├── config.json
│ ├── demo
│ │ ├── demo.vbox
│ │ ├── demo.vbox-prev
│ │ └── Logs
│ │ └── VBox.log
│ ├── disk.vmdk
│ ├── id_rsa
│ ├── id_rsa.pub
│ ├── key.pem
│ ├── server-key.pem
│ └── server.pem
├── demo2
Docker volumes
$ docker volume create --name demo_volume
$ docker run -v demo_volume:/foo ubuntu touch /foo/test
# map to another image
$ docker run -v demo_volume:/bar ubuntu ls -al /bar
# STORAGE PLUGINS
# (flocker, nfs, convoy, glusterfs, rex-ray, ...)
$ docker volume create --driver nfs --name localhost/demo
$ docker run --rm -v localhost/demo:/foo ubuntu ls -al /foo
$ docker inspect -f {{.Volumes}} ecbada822b
map[/data:/var/lib/docker/vfs/dir/cde167197ccc3e138a14f1a4f7c0d65b32cecbada822b0db4cc92e79059437a9]
# touch it from the host
sudo touch /var/lib/docker/vfs/dir/cde167197...37a9/test-file
Docker volumes
# Dockerfile
VOLUME /data
# Data only container + volumes from
$ docker run --name dbdata postgres echo "Data-only container for postgres"
$ docker run -it --volumes-from dbdata debian /bin/bash
# Perform a backup
$ docker run --rm --volumes-from dbdata -v $(pwd):/backup debian \
tar cvf /backup/backup.tar /var/lib/postgresql/data
Docker network
$ docker network
docker: "network" requires a minimum of 1 argument.
See 'docker network --help'.
Usage: docker network [OPTIONS] COMMAND [OPTIONS]
Commands:
ls List all networks
rm Remove a network
create Create a network
connect Connect container to a network
disconnect Disconnect container from a network
inspect Display detailed network information
$ docker network create demo
$ docker network ls
NETWORK ID NAME DRIVER
b7c9b6d0c133 demo bridge
bfc56a57191f none null
0a07e021ffd5 host host
a69e42ce961a bridge bridge
$ docker run --rm -ti \
--name hostA \
debian bash
$ docker run --rm -ti \
--name hostB \
--net demo \
debian bash
$ docker run --rm -ti \
--name hostC \
--net demo \
debian bash
$ ping -c 1 hostB
ping: unknown host
$ ping -c 1 hostC
PING hostC (172.21.0.4): 56 data bytes
64 bytes from 172.21.0.4: icmp_seq=0 ttl=64 time=0.069 ms
--- hostC ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max/stddev = 0.069/0.069/0.069/0.000 ms
$ ping -c 1 hostA
ping: unknown host
Docker network
$ docker network connect demo hostA
$ ping -c 1 hostB
PING hostB (172.21.0.3): 56 data bytes
64 bytes from 172.21.0.3: icmp_seq=0 ttl=64 time=0.069 ms
--- hostB ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max/stddev = 0.069/0.069/0.069/0.000 ms
$ ping -c 1 hostC
PING hostC (172.21.0.4): 56 data bytes
64 bytes from 172.21.0.4: icmp_seq=0 ttl=64 time=0.069 ms
--- hostC ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max/stddev = 0.069/0.069/0.069/0.000 ms
$ ping -c 1 hostA
PING hostA (172.21.0.2): 56 data bytes
64 bytes from 172.21.0.2: icmp_seq=0 ttl=64 time=0.069 ms
--- hostA ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max/stddev = 0.069/0.069/0.069/0.000 ms
$ docker run --rm -ti \
--name hostA \
debian bash
$ docker run --rm -ti \
--name hostB \
--net demo \
debian bash
$ docker run --rm -ti \
--name hostC \
--net demo \
debian bash
Docker network
Docker network
# [app]
# |
# FRONT |
# -----------------[rabbitmq]-----
# BACK |
# |
# [worker]
$ docker network create front
$ docker network create back
$ docker network connect front app
$ docker network connect front rabbitmq
$ docker network connect back rabbitmq
$ docker network connect back worker
Dockerfile
Dockerfile
FROM jpetazzo/dind
MAINTAINER Petr Michalec <epcim@apealive.net>
RUN apt-get update
RUN apt-get install -qqy curl \
sudo \
git \
mercurial \
subversion \
ca-certificates \
locales \
jq
RUN echo 'en_US.UTF-8 UTF-8'>>/etc/locale.gen
RUN locale-gen en_US.UTF-8
ENV LANG en_US.UTF-8
ENV DEBIAN_FRONTEND noninteractive
## CHEF DK ###########################
RUN curl -L https://www.opscode.com/chef/install.sh | sudo bash -s -- -P chefdk
ENV PATH /opt/chefdk/bin:/.chefdk/gem/ruby/2.1.0/bin:/opt/chefdk/embedded/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
# Make Chef DK the primary Ruby/Chef development environment.
RUN echo 'eval "$(chef shell-init bash)"' >> ~/.bash_profile
RUN eval "$(chef shell-init bash)"
RUN chef gem install kitchen-docker
RUN chef gem install kitchen-openstack
RUN chef gem install chef-sugar
RUN chef gem install chef-rewind
RUN chef gem install serverspec
RUN chef gem install infratester
# berks pre-fetch some common soup of cookbooks
RUN mkdir /tmp/fake_cookbook; cd $_
RUN echo "name 'fake_cookbook'\nmaintainer 'fake_cookbook'\nlicense 'fake_cookbook'\ndescription 'fake_cookbook'\nversion '0.0.1'" > metadata.rb
RUN echo "source 'https://supermarket.chef.io'\nmetadata\n\n" > Berksfile
RUN echo "cookbook '7-zip'\ncookbook 'apache2'\ncookbook 'apt'\ncookbook 'ark'\ncookbook 'bluepill'\ncookbook 'build-essential'\ncookbook 'certificate'\ncookbook 'chef-client'\ncookbook 'chef_handler'\ncookbook 'chef_ruby'\ncookbook 'chef-sugar'\ncookbook 'chef-vault'\ncookbook 'cron'\ncookbook 'database'\ncookbook 'device-mapper'\ncookbook 'git'\ncookbook 'minitest-handler'\ncookbook 'modules'\ncookbook 'ncurses'\ncookbook 'nginx'\ncookbook 'ntp'\ncookbook 'ohai'\ncookbook 'openssh'\ncookbook 'openssl'\ncookbook 'packagecloud'\ncookbook 'pacman'\ncookbook 'perl'\ncookbook 'rbenv'\ncookbook 'readline'\ncookbook 'resolver'\ncookbook 'resource-control'\ncookbook 'rsyslog'\ncookbook 'ruby'\ncookbook 'ruby_build'\ncookbook 'runit'\ncookbook 'subversion'\ncookbook 'sudo'\ncookbook 'sysctl'\ncookbook 'system'\ncookbook 'ulimit'\ncookbook 'users'\ncookbook 'windows'\ncookbook 'xml'\ncookbook 'yum'\ncookbook 'yum-epel'\ncookbook 'zlib'\n" >> Berksfile
RUN chef exec berks install
RUN cd -
## FIX UP'S ##########################
RUN chmod -R 0440 /etc/sudoers
RUN chmod -R 0440 /etc/sudoers.d
# workaround (drone.io has no way yet to modify this image before git clone happens)
RUN git config --global http.sslverify false
VOLUME /var/lib/docker
CMD ["wrapdocker"]
Dockerfile
- FROM
- MAINTAINER
- RUN
- CMD
- LABEL
- EXPOSE
- ENV
- ADD
- COPY
- ENTRYPOINT
- VOLUME
- USER
- WORKDIR
- ARG
- ONBUILD
- STOPSIGNAL
Dockerfile Build
$ docker build -t myapp .
Sending build context to Docker daemon 92.67 kB
Step 1 : FROM ruby:2.3.0
---> 7ca70eb2dfea
Step 2 : MAINTAINER luka.dornhecker@gmail.com
---> Running in 61ffc257656d
---> 824c72ddfb87
Removing intermediate container 61ffc257656d
Step 3 : RUN apt-get update -qq && apt-get install -y build-essential nodejs && apt-get clean && rm -rf /var/lib/apt/lists/*
---> Running in 3ee402db540d
Reading package lists...
[...]
---> 445160b5bebb
Removing intermediate container 3ee402db540d
Step 4 : RUN mkdir -p /app
---> Running in 236bd3d78d9a
---> 9441393c6cc8
Removing intermediate container 236bd3d78d9a
Step 5 : WORKDIR /app
---> Running in d29b66063ee8
---> 4e17146f2c38
Removing intermediate container d29b66063ee8
Step 6 : COPY Gemfile Gemfile.lock /app/
---> fcdc11b05832
Removing intermediate container ffb8773a8acb
Step 7 : RUN gem install bundler && bundle install --jobs 20 --retry 5
---> Running in fbeaf1f023b2
Successfully installed bundler-1.11.2
1 gem installed
[...]
Removing intermediate container fbeaf1f023b2
Step 8 : ADD . /app
---> a53ec59090f6
Removing intermediate container e80472550747
Step 9 : ENV PORT 3000
---> Running in 8d68e64db143
---> bceed89e3c6e
Removing intermediate container 8d68e64db143
Step 10 : EXPOSE $PORT
---> Running in 1821fcb3e3d7
---> 8ea96dc0b9f8
Removing intermediate container 1821fcb3e3d7
Step 11 : CMD bundle exec rails -p $PORT
---> Running in f841cd21df61
---> bb540b0da5dc
Removing intermediate container f841cd21df61
Successfully built bb540b0da5dc
Docker compose
$ docker-compose
Define and run multi-container applications with Docker.
Usage:
docker-compose [-f=<arg>...] [options] [COMMAND] [ARGS...]
docker-compose -h|--help
Commands:
build Build or rebuild services
help Get help on a command
kill Kill containers
logs View output from containers
pause Pause services
port Print the public port for a port binding
ps List containers
pull Pulls service images
restart Restart services
rm Remove stopped containers
run Run a one-off command
scale Set number of containers for a service
start Start services
stop Stop services
unpause Unpause services
up Create and start containers
migrate-to-labels Recreate containers to add labels
version Show the Docker-Compose version information
Docker compose
version: '2'
services:
db:
image: postgres
web:
build: .
command: python manage.py runserver 0.0.0.0:8000
volumes:
- .:/code
ports:
- "8000:8000"
depends_on:
- db
app:
build: docker/app
volumes:
- .:/srv
net: app
db:
image: postgres
net: app
app:
build: docker/app
volumes:
- .:/srv
db:
image: postgres
OR
$ docker-compose --x-networking up
Docker compose
Docker compose
version: '2'
networks:
proxy-tier:
external:
name: nginx-proxy
services:
db:
image: postgres
web:
build: .
command: python manage.py runserver 0.0.0.0:8000
volumes:
- .:/code
ports:
- "8000:8000"
depends_on:
- db
lb:
image: tutum/haproxy
links:
- web
ports:
- "80:80"
environment:
- BACKEND_PORT=8000
- BALANCE=roundrobin
compose: scale up
docker-compose up -d
docker-compose ps
docker-compose logs
# Scale up
docker-compose scale web=5
docker-compose up --force-recreate -d
with let's encrypt
version: '2'
networks:
proxy-tier:
external:
name: nginx-proxy
services:
db:
image: postgres
web:
build: .
command: python manage.py runserver 0.0.0.0:8000
volumes:
- .:/code
ports:
- "8000:8000"
depends_on:
- db
nginx:
image: nginx
container_name: nginx
ports:
- "80:80"
- "443:443"
volumes:
- "/etc/nginx/conf.d"
- "/etc/nginx/vhost.d"
- "/usr/share/nginx/html"
- "../../../volumes/proxy/certs:/etc/nginx/certs:ro"
networks:
- proxy-tier
nginx-gen:
image: jwilder/docker-gen
container_name: nginx-gen
volumes:
- "/var/run/docker.sock:/tmp/docker.sock:ro"
- "../../../volumes/proxy/templates/nginx-compose-v2.tmpl:/etc/docker-gen/templates/nginx.tmpl:ro"
volumes_from:
- nginx
entrypoint: /usr/local/bin/docker-gen -notify-sighup nginx -watch -only-exposed -wait 5s:30s /etc/docker-gen/templates/nginx.tmpl /etc/nginx/conf.d/default.conf
letsencrypt-nginx-proxy-companion:
image: jrcs/letsencrypt-nginx-proxy-companion
container_name: letsencrypt-nginx-proxy-companion
volumes_from:
- nginx
volumes:
- "/var/run/docker.sock:/var/run/docker.sock:ro"
- "../../../volumes/proxy/certs:/etc/nginx/certs:rw"
environment:
- NGINX_DOCKER_GEN_CONTAINER=nginx-gen
Prerequisites check
Lab 0.
![](https://s3.amazonaws.com/media-p.slid.es/uploads/81113/images/2311164/Infrastructure.png)
# enable completition
https://github.com/docker/machine/\
blob/master/contrib/completion/\
bash/docker-machine.bash
Dont forget DevOps?
Cooperation
https://????.slack.com/messages/devops-workshop/
- Login with your company email
- Promote your progress, ask others, share code
Docker Lab I.
Break
(~ 5 min)
Docker practice
FIXME
Docker practice
Cross host dependencies
- Ambassador pattern
- link to local container, that act as forwarder
- changing backend service = restarting ambasador
- Environment pattern
- pass the details of remote service as environment variable
- docker -e DB=<IP ADDRESS>
- --env-file <file>
Docker practice
Service Discovery
- consul
- registrator
- etcd
Docker practice
![](https://s3.amazonaws.com/media-p.slid.es/uploads/81113/images/2325340/consul-registrator2.png)
Docker practice
Service Discovery
- consul
- registrator
- etcd
![](https://s3.amazonaws.com/media-p.slid.es/uploads/81113/images/2325341/overview2.png)
Docker practice
Service Discovery
- consul
- registrator
- etcd
![](https://s3.amazonaws.com/media-p.slid.es/uploads/81113/images/2325337/Overview_of_Swarm.png)
Docker Orchestration
Orchestration
- Kubernetes
- Swarm
- Messos framewok
- ...
Docker at Watson
(Martin Cochner)
Docker Lab II.
Review
Questions
Feedback
Break
(~ 5 min)
Let's rock
Hack Session
Backup slides
ELK
![](https://s3.amazonaws.com/media-p.slid.es/uploads/81113/images/2325339/ha-lb-indexer.png)
Grafana & Sensu
![](https://s3.amazonaws.com/media-p.slid.es/uploads/81113/images/2325338/overall.png)
Grafana dashboards
![](https://s3.amazonaws.com/media-p.slid.es/uploads/81113/images/2015338/Grafana.png)
Docker workshop - LEVEL DEVil
By Petr Michalec
Docker workshop - LEVEL DEVil
Docker, intro, basics, labs + hack session. LEVEL DEVil workshop series.
- 2,374