Ansible with Django. Examples for lazy people.

 

by Kamil Gałuszka from S4F

 

Kato.py #2 

Hackerspace Silesia

Feb 2016

Deployment

  • We should have everything automated
  • Making new production server shouldn't take any steps that involves you accessing any server by hand
  • Building new production infrastructure should be possible in hours

Cloud 

IaaS vs PaaS

  • In most cases PaaS isn't sufficient to work with
  • IaaS clouds have more possibilities in the end
  • IaaS is cheaper
  • Handling deployment isn't that hard as it was 10 years ago.
  • PaaS is probably more secure, but less flexible
  • You can always build simple PaaS cloud like Dokku

Ansible
Basics
1. Tasks

Tasks

- name: Installing git
  sudo: true 
  shell: apt-get install git

Tasks

- name: Installing git
  sudo: true 
  apt: name=git state=present

Tasks

- name: Installing git
  sudo: true 
  apt: name=git state=present update_cache=yes upgrade=yes


# update_cache=yes
# apt-get update

# upgrade=yes
# apt-get upgrade mercurial

Tasks

- name: Update cache
  sudo: true 
  apt: update_cache=yes upgrade=yes cache_valid_time=3600

- name: Installing common packages
  sudo: true 
  apt: name={{item}} state=present
  with_items:
    - python
    - ruby
    - git

Tasks

- name: Add PostgreSQL repository entry
  sudo: true
  apt_repository: 
    repo: 'deb http://apt.postgresql.org/pub/repos/apt/ trusty-pgdg main' 
    state: present

- name: Add PostgreSQL repository key
  sudo: true
  apt_key: 
    url: https://www.postgresql.org/media/keys/ACCC4CF8.asc 
    state: present

- name: Install PostgreSQL
  sudo: true
  apt: name=postgresql-9.5 state=present update-cache=yes

Tasks

- name: Install packages
  sudo: True
  npm: name={{item}} global=yes
  with_items:
  - bower
  - gulp

- name: Bower modules install
  bower: path=/home/ubuntu/release/prod_app/bower.json

- name: Compile gulp styles
  command: gulp sass chdir={{paths.app}}

Tasks

  - shell: npm --help
    register: result
    ignore_errors: True

  - npm: package=bower global=yes state=present
    when: result|succeeded

  - command: /bin/something_else
    when: result|failed

  - command: /bin/still/something_else
    when: result|skipped

Ansible
Basics
2. Vars

Vars

# vars/development.yml

app:
  database: my_awesomedatabase


# roles/postgresql/tasks/main.yml

- name: Create postgreSQL database
  sudo: true
  sudo_user: postgres
  postgresql_db: name={{app.database}}
  when: app.rds is undefined

Ansible
Basics
3. Roles

Tasks


- name: Roles is repository of
  items: 
    - tasks
    - handlers
    - files
    - templates

- name: Requirements
  items:
    - tasks should have main.yml as entry file
    - handlers should have main.yml as entry file

- name: Include is simple just put in main.yml file:

- include: tasks/sometasks.yml

Roles

# roles/djangoapp/handlers/main.yml

- name: restart circus
  sudo: True
  service: name=circus state=restarted enabled=yes

# roles/djangoapp/tasks/main.yml

- name: Compile settings to app
  template: 
    src: settings_prod.j2 
    dest: {{ paths.settings }}/{{ app.settings_filename }}
  notify: restart circus

Roles

# roles/djangoapp/templates/settings_prod.j2
DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.postgresql_psycopg2',
        'NAME': '{{ app.database }}',
        'USER': '{{ app.database }}',
        'PASSWORD': '{{ postgresql_password }}',
        'HOST': '{{ postgresql_host }}',
        'PORT': '{{ postgresql_port }}',
        'CONN_MAX_AGE': 3600,
    }
}

Roles

# roles/djangoapp/tasks/main.yml

- name: Copy circusd.ini to /etc/circus/
  sudo: True
  copy: src=circus.ini dest=/etc/circus/circusd.ini owner=root group=root mode=0644
  notify: restart circus

# roles/djangoapp/files/circus.ini

[circus]
endpoint = tcp://127.0.0.1:5555
loglevel = ERROR
logoutput = /var/log/circusd.log
include = app_\*.ini
include_dir = /etc/circus/

Ansible
Basics
4. Playbook

Playbooks

---
- name: apply common configuration to all nodes
  hosts: all
  remote_user: ubuntu
  vars_files:
    - "vars/all.yml"
    - "vars/dev.yml"
  roles:
    - common
    - postgresql
    - nginx
    - redis
    - nodejs
    - djangoapp

Ansible
Basics
5. Vault

Vault

ansible-vault edit vars/prod-pass.yml

Vault

aws_access_key: ADHSSAHDAKDADSADA
aws_secret_key: kjaksdaASDGNt536sRWRGASDAD4wfadaasda
newrelic_key: fadsa35235daafad52352523532534

Ansible
Basics
6. Structure

File structure

- provisioning
  - hosts
    - hosts
    - hosts_dev
  - roles
    - djangoapp
      - files
      - handlers
      - tasks
      - templates
    - elasticsearch
      - tasks
  - vars
    - all.yml
    - dev.yml
    - prod.yml
    - app-pass.yml
  - playbook.yml
  - playbook_dev.yml

So how this works with 

(venv)➜ ✗ pip install ansible 
Installing collected packages: ansible
Successfully installed ansible-2.0.0.2
(venv)➜ ✗ ansible-playbook provisioning/playbook.yml -i provisioning/hosts/hosts 
--ask-vault-pass -vv
Vault password: 
1 plays in provisioning/playbook.yml

PLAY [apply common configuration to all nodes] *********************************
TASK [setup] *******************************************************************
ok: [webapp]
TASK [common : Update existing packages] ***************************************
ok: [webapp] => {"changed": false, "msg": "Reading package lists...
Building dependency tree...
Reading state information...
Reading extended state information...
Initializing package states...
No packages will be installed, upgraded, or removed.
0 packages upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Need to get 0 B of archives. After unpacking 0 B will be used.
Reading package lists...\nBuilding dependency tree...\nReading state information...\nReading extended state information...\nInitializing package states...\n", "stderr": "", "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nReading extended state information...\nInitializing package states...\nNo packages will be installed, upgraded, or removed.\n0 packages upgraded, 0 newly installed, 0 to remove and 0 not upgraded.\nNeed to get 0 B of archives. After unpacking 0 B will be used.\nReading package lists...\nBuilding dependency tree...\nReading state information...\nReading extended state information...
Initializing package states...\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "Reading extended state information...", "Initializing package states...", "No packages will be installed, upgraded, or removed.", "0 packages upgraded, 0 newly installed, 0 to remove and 0 not upgraded.", "Need to get 0 B of archives. After unpacking 0 B will be used.", "Reading package lists...", "Building dependency tree...", "Reading state information...", "Reading extended state information...", "Initializing package states..."]}

PLAY RECAP *********************************************************************
webapp                     : ok=5    changed=0    unreachable=0    failed=0   

Circus

- name: Copy circusd.ini to /etc/circus/
  sudo: True
  copy: src=circus.ini dest=/etc/circus/circusd.ini owner=root group=root mode=0644
  notify: restart circus

- name: Compile circus_app.ini to /etc/circus/
  sudo: True
  template: src=circus_app.ini dest=/etc/circus/app_{{ app.name }}.ini owner=root group=root mode=0644
  notify: restart circus

- name: Compile settings to app
  template: src=settings_prod.j2 dest={{ paths.settings }}/{{ app.settings_filename }}
  notify: restart circus

settings.py

import os
import raven

from .settings import Local

class Live(Local):
    DEBUG = False
    TEMPLATE_DEBUG = DEBUG

    ADMINS = [
        ('Kamil Galuszka', 'kamil.galuszka@solution4future.com'),
    ]

    DATABASES = {
        'default': {
            'ENGINE': 'django.db.backends.postgresql_psycopg2',
            'NAME': '{{ app.database }}',
            'USER': '{{ app.database }}',
            'PASSWORD': '{{ postgresql_password }}',
            'HOST': '{{ postgresql_host }}',
            'PORT': '{{ postgresql_port }}',
            'CONN_MAX_AGE': 3600,
        }
    }

    ALLOWED_HOSTS = ['127.0.0.1', 'localhost', {% for domain in app.domains %}'{{ domain }}', {% endfor %}]

    SERVER_EMAIL = 'server@cluball.com'
    SITE_PROTOCOL = '{{ app.site_protocol }}'

    SITE_URL = 'http://{{ app.domains[0] }}'

    DEFAULT_FILE_STORAGE = 'clubble.storage.MediaRootS3BotoStorage'
    STATICFILES_STORAGE = 'clubble.storage.StaticRootS3BotoStorage'

    AWS_QUERYSTRING_AUTH = False
    import boto.s3.connection

    EMAIL_BACKEND = "django.core.mail.backends.smtp.EmailBackend"

    EMAIL_HOST = 'smtp.mandrillapp.com'
    EMAIL_PORT = 587
    EMAIL_HOST_PASSWORD = "{{ mandrill_key }}"
    EMAIL_USE_TLS = True


    AWS_S3_CALLING_FORMAT = boto.s3.connection.SubdomainCallingFormat()

    AWS_ACCESS_KEY_ID = '{{ aws_access_key }}'
    AWS_SECRET_ACCESS_KEY = '{{ aws_secret_key }}'
    AWS_STORAGE_BUCKET_NAME = '{{ app.aws_s3_bucket }}'
    AWS_TRANSCODE_PIPELINE = '{{ app.aws_s3_pipeline }}'

    STATIC_ROOT = '/static/'
    MEDIA_ROOT = '/media/'
    MEDIA_URL = 'https://{{ app.aws_s3_bucket }}.s3.amazonaws.com/media/'
    STATIC_URL = 'https://{{ app.aws_s3_bucket }}.s3.amazonaws.com/static/'
    RAVEN_CONFIG = {
        'dsn': '{{ raven_url }}',
    }

    SENTRY_CLIENT = 'raven.contrib.django.raven_compat.DjangoClient'

    Local.INSTALLED_APPS += ('raven.contrib.django.raven_compat',)
    IOS_USE_SANDBOX = False


config_file = os.environ.get('NEW_RELIC_CONFIG_FILE')
environment = os.environ.get('NEW_RELIC_ENVIRONMENT')

if config_file and environment:
    newrelic.agent.initialize(config_file, environment)

app_{{name}}.ini

{% macro default_env(label) %}
[env:{{ label }}]
DJANGO_CONFIGURATION = {{ app.configuration }}
DJANGO_SETTINGS_MODULE = {{ app.settings }}
PYTHONPATH={{ paths.source }}:{{ paths.app }}
{% endmacro %}
[watcher:{{app.name}}]
use_sockets = True
numprocesses = {{ app.circus_workers }}
copy_env = True
copy_path = True

uid = {{app.user}}
gid = {{app.group}}
cmd = {{ paths.venv }}/bin/chaussette \
           --fd $(circus.sockets.{{app.name}}) {{app.name}}.wsgi:application
virtualenv = {{ paths.venv }}
working_dir = {{ paths.app }}

{{ default_logs(app.name) }}
{{ default_env(app.name) }}

[socket:{{app.name}}]
port = {{ app.circus_port }}
host = 127.0.0.1

Vagrant + Ansible

# -*- mode: ruby -*-
# vi: set ft=ruby :

Vagrant.configure(2) do |config|
  config.vm.box = "ubuntu/trusty64"

  config.vm.synced_folder ".", "/vagrant"

  config.vm.provision "ansible" do |ansible|
    ansible.playbook = "provisioning/playbook_vagrant.yml"
    ansible.ask_vault_pass = true
    ansible.verbose = "vvv"
    ansible.raw_arguments = ['--extra-vars=provisioning/vars/prod-pass.yml']

  end

end

Thanks! Any questions?

Ansible with Django

By Kamil Gałuszka

Ansible with Django

  • 1,009
Loading comments...

More from Kamil Gałuszka