Shipping Search

Running elasticsearch in Docker Containers

Let's start filling your head!

(Short) History

• A bunch of .go files, released 2013-01-19 on github
• A global OSS community and enterprise SaaS/PaaS hype since summer 2014
• IANA-assigned ports 2375 and 2376 in 2014
• Supported by every modern Linux distro
• Supported by lots of $money from major tech companies

What does Docker do?

Vs

OS vs. Process Virtualization

Inside a Container

• Run multiple Linux distros in the same Kernel space
  on your docker host system, meaning:
  • Low memory consumption
  • Quick startup time for new container instances
  • Production and development environments can be exactly the same (finally!)
• Every container is based on an immutable filesystem image and the changes needed to run your app get stacked and isolated on top of this base image using aufs
• Your application lives in a single container, based on an image
• Your application's dependencies like DBs and stuff run in other containers. They are interlinked at runtime with --link
• Every part of the stack is cached on the docker host to make starting, stopping and building containers fast

Some Dockerspeak

• What is an image?
  • A portable blueprint for your container
    based on a single textfile called Dockerfile
• What is a container?
  • A running Linux server based on an image which is
    usually executing a single application in a small
    pidspace and may be interlinked with other containers
• What is a volume?
  • Space on disk which gets mounted to one or multiple containers during their runtime for data persistence
• What is a Docker registry?
  • Github for Dockerfiles
• What is Github?

Mindset

• Don't think of virtualization, think of running processes
• Containers are tiny and should only run one process
• Software in containers shouldn't be updated at runtime but replaced by a new version (=immutable deployments)
• Containers are isolated from each other by default
• Build complex stacks by interlinking containers
• Data is stored outside the container on the Docker host
  • Exception: Container Volumes
• Containers can be exported and imported as tarfiles. With docker, you just deal with files, not with disk images
  • docker cp jenkins_container:/var/jenkins_home - \
    >./jenkins_home.tar.gz

Lets run...

Python 2 and 3

$> docker run -i -t python:2
$> docker run -i -t python:3

Note:

1. We just instantiated a Docker container on our remote Docker host based on an Docker image for Python
2. Our local `docker` client attached an interactive pseudo-TTY for in and output
3. Strg+c kills the container

Start a module

$> docker run -it -p 8888:8888 \ python:3 python -m http.server 8888
$> curl -i <server>:8888

1. We just instantiated a Docker container on our remote Docker host based on an Docker image for Python
2. Our local `docker` client attached an interactive pseudo-TTY for in and output
3. We overwrite the default CMD with our own command

A bit of introspection

$> docker run -it -d --name utnubu ubuntu
$> docker inspect utnubu | jq '.'
$> docker stats $(docker ps -l -q)
$> docker exec -it utnubu /bin/bash
$> # ps, top, uname -a, apt-get, what you want

1. We just instantiated a Docker container on our remote Docker host based on an Docker image for Ubuntu
2. We check some information about the container

Lets run elasticsearch

We use the official Docker-Hub image

Lets have a look at the image

• Look into the latest Dockerfile

• Grab the Container

  • docker pull elasticsearch

• Start a node and expose the ports

  • docker run --name trick -d -p 9200:9200 \
    -p 9300:9300 elasticsearch

• Check the HTTP interface

  • curl -L http://$(docker-machine ip):9200​

Do more

• Run a second container and link it to the first

  • docker run --name track --link trick -d elasticsearch

• Check if we can see our second node

  • curl http://$(docker-machine ip):9200/_cluster/health

• Let's git clone elasticsearch-head and connect to the cluster

• Anything else?

The good

• User friendly & lightweight

• Solves the dependency hell

• Reusability by design

• Client/server RESTful communication API

• Promotes github-like community and collaboration through a public registry

• Data in- and output is
  (tar-)file based

• Massive, growing ecosystem

The bad (IMHO)

• Dockerfiles feel restrictive and dowdy compared to ansible playbooks for example

• No orchestration built in, but the situation is getting better

• The Docker volumes are an afterthought, not a first class citizen (yet?) and not every subcommand can deal with them

• Linux only

• Security (privilege-escalation)