Downsizing the DC

How to fit your production environment into a single laptop

.meta

  • Ricardo Kirkner (@ricardokirkner)
  • Working with Python since 2004
  • Like to play with innovative technologies

Topics

  • Typical Infrastructure of a modern application
  • How can we manage this infrastructure?
    • The classic way
    • The modern way
  • Development workflow

Infrastructure

Application

Application

Database

Web Server

Application

WSGI Server

Database

Web Server

Application

WSGI Server

Database

Cache

Task Queue

Email Server

Application

WSGI Server

Database

Cache

Task Queue

Email Server

Web Server

Frontend Cache

Load Balancer

Application

WSGI Server

Database

Cache

Task Queue

Email Server

Logging

Monitoring

Metrics

Web Server

Frontend Cache

Load Balancer

Managing the infrastructure

  • Scripts
  • Configuration Management
  • ???

Using scripts

# provision.sh

apt-get install apache2
apt-get install postgresql
apt-get install python-psycopg2
...
# configure.sh

cp config/apache/site.conf /etc/apache2/sites-available/site.conf
cp config/postgresql/pg_hba.conf /etc/postgresql/pg_hba.conf

a2dissite default
a2ensite site

createuser dbuser
createdb -E utf-8 -O dbuser dbname

service apache2 restart
service postgresql restart

Configuration Management

  • Puppet
  • Chef
  • Ansible

didn't you mention something about available hardware?

but first...

Web Server

Application

WSGI Server

Database

Cache

Task Queue

Email Server

Logging

Monitoring

Metrics

Frontend Cache

Load Balancer

Application

WSGI Server

Database

Cache

Task Queue

Email Server

Logging

Monitoring

Metrics

Web Server

Frontend Cache

Load Balancer

Application

WSGI Server

Database

Cache

Task Queue

Email Server

Logging

Monitoring

Metrics

Web Server

Frontend Cache

Load Balancer

Web Server

Application

WSGI Server

Database

Cache

Task Queue

Email Server

Logging

Monitoring

Metrics

Frontend Cache

Load Balancer

Application

WSGI Server

Database

Cache

Task Queue

Email Server

Logging

Monitoring

Metrics

Web Server

Frontend Cache

Load Balancer

11 Servers

1 Laptop

!=

Now what?

Shrink the DC?

Enter

docker-compose

web:
  build: .
  ports:
   - "8000:8000"
  volumes:
   - .:/code
  links:
   - redis
   - db

redis:
  image: redis
  expose:
    - "6379"

db:
  image: postgresql
  expose:
    - "5432"

docker-compose

$ docker-compose build
$ docker-compose up -d
$ docker-compose ps
$ docker-compose logs
$ docker-compose scale app=4 web=2
$ docker-compose stop
$ docker-compose rm

docker-compose


$ vim app/app.py

$ docker-compose build

$ docker-compose up

$ firefox http://localhost:8000/

Case Study

Step 1: Minimal Application

Application

.
├── logs
├── Makefile
├── manage.py
├── project
│   ├── __init__.py
│   ├── settings.py
│   ├── urls.py
│   └── wsgi.py
├── README.md
└── requirements.txt

2 directories, 8 files

Step 1: Minimal Application

# Development setup

$ make bootstrap

$ make manage ARGS='syncdb --noinput'

$ make run

Step 1: Minimal Application

# Dockerfile

FROM python:3-onbuild
MAINTAINER Ricardo Kirkner <ricardo@kirkner.com.ar>

WORKDIR /usr/src/app
CMD ["make", "start"]

Step 2: Dockerize Application

# Makefile

start:
        python manage.py syncdb --noinput
        python manage.py migrate --noinput
        python manage.py runserver 0.0.0.0:8000

.
├── app
│   ├── Dockerfile
│   ├── logs
│   ├── Makefile
│   ├── manage.py
│   ├── project
│   │   ├── __init__.py
│   │   ├── settings.py
│   │   ├── urls.py
│   │   └── wsgi.py
│   ├── README.md
│   └── requirements.txt
├── build-requirements.txt
├── docker-compose.yml
├── logs
└── Makefile

Step 3: Define stack

# docker-compose.yml

app:
    build: app
    volumes:
        - app:/usr/src/app
    ports:
        - "8000:8000"

Step 3: Compose stack

Step 4: Add a PostgreSQL database

Application

Database

# docker-compose.yml

app:
    build: app
    volumes:
        - app:/usr/src/app
    ports:
        - "8000:8000"
    links:
        - db
    environment:
        DATABASE_URL: postgres://postgres@db/postgres
db:
    image: postgres
    expose:
        - "5432"

Step 4: Add a PostgreSQL database

# app/project/settings.py

 DATABASES = {
-    'default': {
-        'ENGINE': 'django.db.backends.sqlite3',
-        'NAME': os.path.join(BASE_DIR, 'db.sqlite3'),
-    }
+    'default': dj_database_url.config(default='sqlite:///db.sqlite3')
 }

Step 4: Add a PostgreSQL database

Step 5: Add a WSGI Server

Application

WSGI Server

Database

# app/Makefile

run: ARGS=-b 0.0.0.0:8000
run:
        $(MANAGE) collectstatic --noinput
        $(GUNICORN) $(ARGS) project.wsgi:application

start:
        python manage.py migrate --noinput
        python manage.py collectstatic --noinput
        gunicorn -b 0.0.0.0:8000 project.wsgi:application

Step 5: Add a WSGI Server

Step 5: Add a Web Server

Web Server

Application

WSGI Server

Database

.
├── app
│   ├── Dockerfile
│   ├── logs
│   ├── Makefile
│   ├── manage.py
│   ├── project
│   │   ├── __init__.py
│   │   ├── settings.py
│   │   ├── urls.py
│   │   └── wsgi.py
│   ├── README.md
│   └── requirements.txt
├── build-requirements.txt
├── docker-compose.yml
├── logs
├── Makefile
└── web
    ├── app.conf
    ├── Dockerfile
    └── Makefile

5 directories, 15 files

Step 5: Add a Web Server

# docker-compose.yml

app:
    build: app
    volumes:
        - app:/usr/src/app
    ports:
        - "8000:8000"
    links:
        - db
    environment:
        DATABASE_URL: postgres://postgres@db/postgres
db:
    image: postgres
    expose:
        - "5432"
web:
    build: web
    ports:
        - "80:80"
    links:
        - app

Step 5: Add a Web Server

# Dockerfile

FROM nginx
MAINTAINER Ricardo Kirkner <ricardo@kirkner.com.ar>

ADD app.conf /etc/nginx/conf.d/default.conf

Step 5: Add a Web Server

Step 5: Add a Web Server

# web/app.conf

upstream appserver {
    server app:8000;
}

server {
    listen 80;

    location / {
        proxy_pass http://appserver;
        proxy_redirect off;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Host $server_name;
    }
}

Step 6a: Using a cache

.
├── app
│   ├── Dockerfile
│   ├── hits
│   │   ├── admin.py
│   │   ├── __init__.py
│   │   ├── migrations
│   │   │   └── __init__.py
│   │   ├── models.py
│   │   ├── templates
│   │   │   └── hits
│   │   │       └── hitme.html
│   │   ├── tests.py
│   │   └── views.py

Step 6a: Using a cache

# app/hits/views.py

from django.core.cache import caches
from django.shortcuts import render


def hitme(request):
    cache = caches['hits']
    try:
        hits = cache.incr('hits')
    except ValueError:
        cache.set('hits', 1)
        hits = 1
    return render(request, 'hits/hitme.html', {'hits': hits})

Step 6a: Using a cache

# app/project/settings.py

#########
# CACHE #
#########

CACHES = {
    'default': {
        'BACKEND': 'django.core.cache.backends.locmem.LocMemCache',
    },
    'hits': {
        'BACKEND': 'django.core.cache.backends.locmem.LocMemCache',
    }
}

Step 6b: Using a production level cache

Web Server

Application

WSGI Server

Database

Cache

Step 6b: Using a production level cache

# app/project/settings.py

#########
# CACHE #
#########

CACHES = {
    'default': {
        'BACKEND': 'django.core.cache.backends.locmem.LocMemCache',
    },
    'hits': {
        'BACKEND': 'django_redis.cache.RedisCache',
        'LOCATION': env('REDIS_URL', 'redis://127.0.0.1:6379/1'),
        'OPTIONS': {
            'CLIENT_CLASS': 'django_redis.client.DefaultClient',
        }
    }
}

Step 6b: Using a production level cache

# docker-compose.yml

app:
    build: app
    volumes:
        - app:/usr/src/app
    ports:
        - "8000:8000"
    links:
        - db
        - redis
    environment:
        DATABASE_URL: postgres://postgres@db/postgres
        REDIS_URL: redis://redis/1
redis:
    image: redis
    expose:
        - "6379"

Step 7: Scheduling asynchronous tasks

Web Server

Application

WSGI Server

Database

Cache

Task Queue

Step 7: Scheduling asynchronous tasks

.
├── app
...   
│   ├── lottery
│   │   ├── admin.py
│   │   ├── __init__.py
│   │   ├── migrations
│   │   │   └── __init__.py
│   │   ├── models.py
│   │   ├── tasks.py
│   │   ├── templates
│   │   │   └── lottery
│   │   │       └── lottery.html
│   │   ├── tests.py
│   │   └── views.py
...
│   ├── Procfile
│   ├── project
│   │   ├── celery.py
...

Step 7: Scheduling asynchronous tasks

# app/Procfile

web: gunicorn -b 0.0.0.0:8000 project.wsgi:application
worker: celery -A project worker -l info

# app/Makefile

 start:
        python manage.py migrate --noinput
        python manage.py collectstatic --noinput
-       gunicorn -b 0.0.0.0:8000 project.wsgi:application
+       honcho start
# app/project/settings.py

##########
# CELERY #
##########

BROKER_URL = env('BROKER_URL', 'amqp://guest:guest@localhost/')
CELERY_RESULT_SERIALIZER = 'json'
CELERY_ACCEPT_CONTENT = ['json']
CELERY_TASK_SERIALIZER = 'json'

Step 7: Scheduling asynchronous tasks

# docker-compose.yml

app:
    build: app
    volumes:
        - app:/usr/src/app
        - data/logs/app:/usr/src/app/logs
    ports:
        - "8000:8000"
    links:
        - db
        - redis
        - amqp
    environment:
        DATABASE_URL: postgres://postgres@db/postgres
        REDIS_URL: redis://redis/1
        BROKER_URL: amqp://guest:guest@amqp/
amqp:
    image: rabbitmq
    expose:
        - "5672"

Step 8: Monitoring asynchronous tasks

Web Server

Application

WSGI Server

Database

Cache

Task Queue

Monitoring

Step 8: Monitoring asynchronous tasks

# docker-compose.yml

amqp:
    image: rabbitmq:management
    volumes:
        - data/logs/amqp:/var/log/rabbitmq
    expose:
        - "5672"
        - "15672"
flower:
    image: nicholsn/docker-flower
    links:
        - amqp:mq
    ports:
        - "5555:5555"

Step 9: Metrics

Web Server

Application

WSGI Server

Database

Cache

Task Queue

Monitoring

Metrics

Step 9: Metrics

# docker-compose.yml

app:
    ...
    links:
        - db
        - redis
        - amqp
        - graphite
    environment:
        ...
        STATSD_HOST: graphite
graphite:
    image: hopsoft/graphite-statsd
    expose:
        - "80"
        - "2003"
        - "8125"
    ports:
        - "8080:80"

Step 9: Metrics

# app/project/settings.py

 INSTALLED_APPS = (
     ...
     'hits',
     'lottery',
+    'django_statsd',
 )

 MIDDLEWARE_CLASSES = (
+    'django_statsd.middleware.StatsdMiddleware',
     'django.contrib.sessions.middleware.SessionMiddleware',
     ...
     'django.middleware.security.SecurityMiddleware',
+    'django_statsd.middleware.StatsdMiddlewareTimer',
 )

##########
# STATSD #
##########

STATSD_HOST = env('STATSD_HOST', 'localhost')

Step 10: Scaling

Web Server

Application

WSGI Server

Database

Cache

Task Queue

Monitoring

Metrics

Frontend Cache

Load Balancer

Step 10: Scaling

# docker-compose.yml

consul:
    image: progrium/consul
    ports:
        - "8500:8500"
    command: -server -bootstrap -advertise 172.17.0.1

registrator:
    image: gliderlabs/registrator
    command: consul://172.17.0.1:8500
    volumes:
        - "/var/run/docker.sock:/tmp/docker.sock"
    links:
        - consul

Step 10: Scaling

# docker-compose.yml

app:
    ...
    links:
        ...
        - consul
    environment:
        ...
        SERVICE_8000_NAME: "app"
        SERVICE_8000_TAGS: "backend"

web:
    ...
    links:
        - consul

Step 10: Scaling

# nginx.tmpl

...
upstream app {
    {{range service "backend.app"}}
    server 172.17.0.1:{{.Port}};
    {{end}}
}
...
# Dockerfile

...
COPY nginx.tmpl ./
CMD service nginx start && \
    /opt/consul-template-nginx/consul-template \
    -consul consul:8500 \
    -template '/opt/consul-template-nginx/nginx.tmpl:\
               /etc/nginx/nginx.conf:service nginx reload'

Demo

Thank you!

Questions?

https://github.com/ricardokirkner/docker-devel-pycon-ar-2015

https://slides.com/ricardokirkner/docker-devel-pycon-ar-2015

Made with Slides.com