Devops

with Django Docker & Gitlab

whoami

Luke Spademan

0) Student

1) starting A levels

3) gave a lightening talk last year

2) @lukespademan: [twitter, git, etc]

Maths
Further Maths
Computer Science
Physics

Devops?

What is Devops?

Deops is trying to automate stuff you would normaly have to do manually.

Benifits

Saves you time
Stop you making mistakes
Satisfying

Docker?

What is Docker

Docker allows you to containerise your code
Like a stripped down virtual machine
If they have docker installed there should be no issue running your code

Example

Python

pip install pipenv --user

mkdir project
cd project
mkdir src
cd src

pipenv python --3.6  # creates the virtual environment
pipenv install django gunicorn psycopg2-binary  # installs pakcages to venv
pipenv shell  # enteres the virtual environment
django-admin startproject mysite .

django-admin startproject mysite .

Python

# src/mysite/settings.py

# ...

try:
    SECRET_KEY

except NameError:
    SECRET_FILE = os.path.join(BASE_DIR, 'secret.txt')
    try:
        SECRET_KEY = open(SECRET_FILE).read().strip()

    except IOError:
        try:
            import random
            SECRET_KEY = ''.join([random.SystemRandom().choice(
                'abcdefghijklmnopqrstuvxyz0123456789!@#$%^&*(-_=+)') for i in range(50)]
                                                               )

            secret = open(SECRET_FILE, "w")
            secret.write(SECRET_KEY)
            secret.close()

        except IOError:
            Exception("Please create a %s file with random characters." % SECRET_FILE)

# ...

Python

# src/mysite/settings.py

# ...

# SECURITY WARNING: don't run with debug turned on in production!

if os.environ.get('DJANGO_DEBUG'):
    print("Debug is enabled.")
    DEBUG = True
    ALLOWED_HOSTS = ["localhost", "127.0.0.1"]

else:
    DEBUG = False
    ALLOWED_HOSTS = ["example.com", "localhost"]
# ...

Python

# src/mysite/settings.py

# ...

else:
    DATABASES = {
        'default': {
            'ENGINE': 'django.db.backends.postgresql_psycopg2',
            'NAME': 'postgres',
            'USER': 'postgres',
            'HOST': 'db',
            'PORT': 5432,
        }
    }
# ...

if DEBUG == True:
    DATABASES = {
        'default': {
            'ENGINE': 'django.db.backends.sqlite3',
            'NAME': os.path.join(BASE_DIR, 'db.sqlite3'),
        }
    }

Python

# src/mysite/settings.py

# ...

# Static files (CSS, JavaScript, Images)
# https://docs.djangoproject.com/en/2.0/howto/static-files/

STATIC_URL = '/static/'
STATIC_ROOT = os.path.join(BASE_DIR, "static/")

File Tree

.
└── src
   ├── db.sqlite3
   ├── main.py
   ├── manage.py
   ├── mysite
   │   ├── __init__.py
   │   ├── settings.py
   │   ├── urls.py
   │   └── wsgi.py
   ├── Pipfile
   ├── Pipfile.lock
   └── secret.txt

Test

cd src

pipenv shell

DJANGO_DEBUG=True python manage.py makemigrations
DJANGO_DEBUG=True python manage.py migrate

DJANGO_DEBUG=True python manage.py runserver

localhost:8000

Hello, PyCon UK!

hello_world app

python manage.py startapp hello_world

# src/hello_world/urls.py

from django.urls import path
from . import views

urlpatterns = [
    path('', views.index, name='index'),
]

# src/hello_world/views.py

from django.shortcuts import render

def index(request):
    return render(request, 'hello_world/index.html')

<!DOCTYPE html>
<html>
  <head>
    <meta charset="utf-8">
    <title>Hello, PyCon UK!</title>
  </head>
  <body>
    <h1>Hello, PyCon UK!</h1>
  </body>
</html>

# src/hello_world/templates/hello_world/index.html

Update settings.py

# src/mysite/settings.py

...

INSTALLED_APPS = [
    'hello_world',
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
]

...

Update urls.py

# src/mysite/urls.py

from django.contrib import admin
from django.urls import path, include

urlpatterns = [
    path('admin/', admin.site.urls),
    path('', include('hello_world.urls')),
]

Docker

Dockerfile

# Dockerfile

FORM python:3.6

RUN mkdir /src
ADD src/Pipfile /src
ADD src/Pipfile.lock /src

RUN pip install pipenv

WORKDIR /src
RUN pipenv install --system --deploy --ignore-pipfile

ADD src /src

Docker Compose

# docker-compose.yml

version: '3'

services:
  nginx:
    image: registry.gitlab.com/lukespademan/mysite/nginx:latest
    build:
        context: .
        dockerfile: Dockerfile.nginx
    container_name: mysite_nginx
    depends_on:
    - web
    ports:
    - "80:8080"
  web:
    image: registry.gitlab.com/lukespademan/mysite:latest
    build: .
    container_name: mysite_django
    command: bash start_django.sh
    depends_on:
    - db
  db:
    image: postgres:latest
    container_name: mysite_postgres
    volumes:
    - "db:/var/lib/postgresql/data"

volumes:
  db: {}

services:
  nginx:
    build: Dockerfile.nginx
    container_name: mysite_nginx
    depends_on:
    - web
    ports:
    - "80:8080"

  web:
    build: .
    container_name: mysite_django
    command: bash start_django.sh
    depends_on:
    - db

  db:
    image: postgres:latest
    container_name: mysite_postgres
    volumes:
    - "db:/var/lib/postgresql/data"

volumes:
  db: {}

Start Django

# src/start_django.sh

#!/bin/bash

ls
cat Pipfile
pip freeze
python manage.py migrate
python manage.py collectstatic --noinput
gunicorn mysite.wsgi -b 0.0.0.0:8000

Dockerfile.nginx

FROM nginx:latest

ADD config/nginx /etc/nginx/conf.d
ADD src /src
EXPOSE 80

Nginx Config

mkdir config
cd config

mkdir nginx
cd nginx

# config/nginx/my_django.conf

upstream web {
	ip_hash;
	server web:8000;
}

upstream web {
	ip_hash;
	server web:8000;
}

server {
	location /robots.txt {alias /src/static/robots.txt;}
	location /favicon.ico {alias /src/static/favicon.ico;}
	location /static/ {
		autoindex on;
		alias /src/static/;
	}

	location / {
		proxy_pass http://web/;
	}
	listen 8080;
}

File Tree

.
└── config
│  └── nginx
│      └── my_django.conf
├── docker-compose.yml
├── Dockerfile
└── src
   ├── db.sqlite3
   ├── main.py
   ├── manage.py
   ├── mysite
   │   ├── __init__.py
   │   ├── settings.py
   │   ├── urls.py
   │   └── wsgi.py
   ├── Pipfile
   ├── Pipfile.lock
   ├── secret.txt
   └── start_django.sh

Test

sudo docker-compose build

sudo docker-compose up

localhost

Git

.gitignore

# .gitignore

__pycache__
db.sqlite3
secret.txt

REAME.md

# Devops Example

LISENCE

MIT License

Copyright (c) 2018 Luke Spademan

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

Push to repo then

Runner

Get a server
Setup Respository
Install gitlab-runner
Register runner

curl -L https://packages.gitlab.com/install/repositories/runner/gitlab-runner/script.deb.sh | \
    sudo bash

sudo apt-get install gitlab-runner

sudo gitlab-runner register

Runner

# /etc/gitlab-runner/config.toml
...

    volumes = ["cache", "/var/run/docker.sock:/var/run/docker.sock"]
...

Runner

Login
Start the Runner
Start docker swarm

sudo docker login registry.gitlab.com

sudo gitlab-runner start

sudo docker warm init --advertise-addr 123.123.123.123

Disable Shared Runners

.gitlab-ci.yml

image: tmaier/docker-compose:latest

stages:
    - build
    - test
    - deploy

variables:
    WEB_CONTAINER_IMAGE: registry.gitlab.com/lukespademan/mysite:latest
    NGINX_CONTAINER_IMAGE: registry.gitlab.com/lukespademan/mysite/nginx:latest

before_script:
    - docker login -u gitlab-ci-token -p $CI_JOB_TOKEN registry.gitlab.com

.gitlab-ci.yml

test:
    stage: test
    script:
        - docker-compose run web python manage.py test --no-input

build:
    stage: build
    script:
        - docker-compose build
        - docker-compose push
    tags:
        - docker



deploy:
    stage: deploy
    script:
        - docker stack deploy --compose-file docker-compose.yml stack-name
    tags:
        - docker

    only:
        - master

Done?

Testing

Navigate to http://server-domain-name.com

Devops

with Django Docker & Gitlab

whoami

Luke Spademan

Devops?

What is Devops?

Benifits

Docker?

What is Docker

Example

Python

Python

Python

Python

Python

Python

File Tree

Test

localhost:8000

Hello, PyCon UK!

hello_world app

Update settings.py

Update urls.py

Docker

Dockerfile

Docker Compose

Start Django

Dockerfile.nginx

Nginx Config

File Tree

Test

localhost

Git

.gitignore

REAME.md

LISENCE

Push to repo then

Runner

Runner

Runner

Runner

Disable Shared Runners

.gitlab-ci.yml

.gitlab-ci.yml

Done?

Testing

Navigate to http://server-domain-name.com

Devops

More from Luke