Common Code Learns Ansible

I'm really not the expert in this stuff

What is "Ansible"

  • Bullet One
  • Bullet Two
  • Bullet Three

Why Ansible for CC?

  • It's simple and powerful - it's something most people here should be able to learn in a couple of days
  • It has _really_ good documentation
  • Compared to puppet the overhead is minimal
  • We can package it up and give it to a client

common-ansible

https://github.com/commoncode/common-ansible

We use Ansible for application deployment and server management

(keeping shit up to date)

Installation

Python 2 only :(

 

> pip install ansible

Since we're mostly on Python 3 now, it might make sense to install it globally

 

> sudo pip install ansible

Anatomy of common-ansible

  • Hosts (inventory)
    Define where you will execute the ansible playbook

 

  • In common-ansible we've created a hosts directory so that we can keep projects separate
  • Hosts are either FQDNs, IPs or (most likely) entries in your SSH config
  • In common-ansible we can optionally add a server_hostname_suffix variable to hostnames

Anatomy of common-ansible

# hosts/common-api

[common-api-production]
188.166.252.219  # api.common.io



# hosts/bpcle

[bpcle-staging]
bpcle-staging server_hostname_suffix=staging

[bpcle-demo]
demo.bpcletool.net.au server_hostname_suffix=demo

[bpcle-beta]
beta.bpcletool.net.au server_hostname_suffix=beta

[bpcle-production]
52.63.208.183 server_hostname_suffix=production

Anatomy of common-ansible

[healthsite-bastion-hosts]
healthsite-new-gw-ap-southeast-2a.hs.commoncode.com.au server_hostname_suffix=gw-2a
healthsite-new-gw-ap-southeast-2b.hs.commoncode.com.au server_hostname_suffix=gw-2b

[healthsite-web-hosts]
healthsite-web-4 server_hostname_suffix=4
healthsite-web-5 server_hostname_suffix=5
healthsite-web-6 server_hostname_suffix=6
healthsite-web-7 server_hostname_suffix=7
healthsite-web-8 server_hostname_suffix=8

[belvedere]
healthsite-web-5 server_hostname_suffix=5

[blackburnsouth]
healthsite-web-5 server_hostname_suffix=5

[biggerafamilymedical]
healthsite-web-5 server_hostname_suffix=5

[eastadelaide]
healthsite-web-5 server_hostname_suffix=5

[electra]
healthsite-web-5 server_hostname_suffix=5

[eramosa]
healthsite-web-5 server_hostname_suffix=5

[fcdhealth]
healthsite-web-5 server_hostname_suffix=5

[forestroad]
healthsite-web-5 server_hostname_suffix=5

[kincraig]
healthsite-web-5 server_hostname_suffix=5

[medicalhq]
healthsite-web-5 server_hostname_suffix=5

[mornington]
healthsite-web-5 server_hostname_suffix=5

[mygpclinic]
healthsite-web-5 server_hostname_suffix=5

[nillumbik]
healthsite-web-5 server_hostname_suffix=5

[onehealth]
healthsite-web-5 server_hostname_suffix=5

[richmond]
healthsite-web-5 server_hostname_suffix=5

[siaburwood]
healthsite-web-5 server_hostname_suffix=5

[sunnybank]
healthsite-web-5 server_hostname_suffix=5

[southyarra]
healthsite-web-5 server_hostname_suffix=5

[staging]
healthsite-web-5 server_hostname_suffix=5

[towerhill]
healthsite-web-5 server_hostname_suffix=5

[monashlink]
healthsite-web-5 server_hostname_suffix=5

[newcombmedicalcentre]
healthsite-web-5 server_hostname_suffix=5

[baysidefamilymedical]
healthsite-web-7 server_hostname_suffix=7

[parkdalemedicalcentre]
healthsite-web-6 server_hostname_suffix=6

[midvalleyfamilymedicine]
healthsite-web-7 server_hostname_suffix=7

[medicalcentrefrankston]
healthsite-web-7 server_hostname_suffix=7

[yarramedical]
healthsite-web-7 server_hostname_suffix=7

[doctorsofnorthcote]
healthsite-web-8 server_hostname_suffix=8

[siamedicalfootscray]
healthsite-web-6 server_hostname_suffix=6

Anatomy of common-ansible

  • Roles
    The things that will be executed on the remote server 

 

  • We've built up a pretty large library of roles that do things "our way"
  • Roles have: tasks, files, templates, handlers and defaults
  • For each project we define which roles you'd like to execute (and which order to do it in)

Let's look at our nginx role

common-ansible roles

  • Basic server configuration
  • Adding users (and the delivery directors)
  • nginx, gunicorn, upstart, django
  • Postgres on the server
  • Celery, Redis, RabbitMQ, Memcached
  • SSL with Let's Encrypt
  • New Relic monitoring
  • Sentry for your Django app
  • Opbeat monitoring
  • nvm

Anatomy of common-ansible

  • Vars
    The variables that apply to a specific project / environment

 

  • We create a directory for each project, with vars files for each environment
> ls -lh vars/commonio
-rw-r--r--  1 brenton  staff   1.2K May 25 07:24 production.yml
-rw-r--r--  1 brenton  staff   1.2K Apr 29 11:07 staging.yml

Anatomy of common-ansible

  • The vars files define the project and environment specific values
  • i.e. "domain_name", "deploy_version", "repo", "requirements_path"
  • Most variables have defaults (more on that in a moment) - this is where you override them
  • You point at the variables with the playbook

Anatomy of common-ansible

---

app_name: django_camp
repo: git@github.com:melbdjango/django_camp.git
deploy_version: master

env: prod
domain_name: melbdjango.camp
extra_domains: 'www.melbdjango.camp'

server_hostname_base: 'commoncode-web'

opbeat_org_id: 'XXXXXXXX'
opbeat_app_id: 'XXXXXXXX'
opbeat_secret_token: 'XXXXXXXX'

new_relic_key: 'XXXXXXXX'

gunicorn_port: 8010

django_environment:
  DJANGO_DATABASE_NAME: "melbdjangocamp_production"
  DJANGO_DATABASE_HOST: "commonvn3.ap-southeast-2.rds.amazonaws.com"
  DJANGO_DATABASE_USERNAME: "hofstadter"
  DJANGO_DATABASE_PASSWORD: "XXXXXXXX"
  DJANGO_SECRET_KEY: "XXXXXXXX"
  DJANGO_MODE: "Production"
  DJANGO_DEBUG: False

Anatomy of common-ansible

  • Playbooks
    Define the vars, roles and hosts that manage the server configuration

 

  • We keep it pretty damn simple (most of the time)
  • "hosts", "vars_files", "roles"
  • This also defines the user that you'll connect as and become once you're on the server

Playbook for Common API

---

- hosts: common-api-production
  become: yes
  become_user: root
  gather_facts: yes

  vars_files:
    - vars/common-api/production.yml

  roles:
    - base
    - deliverydirectors
    - app
    - letsencrypt
    - postgres
    - nginx
    - upstart
    - django
    - common_api_cron
    - ownership

Anatomy of common-ansible

  • Ansible Vault
    Where we (sometimes) keep the real secret stuff

 

  • ansible-vault allows you to have encrypted "vars" for your project
  • In the Django world this is handy for your SECRET_KEY, AWS keys, etc.
> ansible-vault create my-secrets.yml

> ansible-vault edit --vault-password-file=.bpcle-password-file \
vars/bpcle/production-secrets.yml

Deploying api.common.io

> ansible-playbook -i hosts/common-api common-api.yml
                      ^ hosts file     ^ playbook


> ansible-playbook -i hosts/common-api -t deploy common-api.yml
                                       ^^^ only run tasks with this tag


> ansible-playbook -i hosts/bpcle -l bpcle-production bpcle.yml
                                  ^^^ only run on hosts that match this pattern

Take a moment to let the poorly prepared talk sink in

How you could help

  • We're not using handlers at all and that's dumb
  • DRY up the vars files by moving more into the playbooks or "project" vars files
  • Use a fork of common-ansible for your own projects and let us know how you go
  • Add support for projects that aren't Django project

Common Code Learns Ansible

By Brenton Cleeland

Common Code Learns Ansible

  • 507
Loading comments...

More from Brenton Cleeland