Continuous Delivery
for Python Developers

PyCon 8, 2017

live slides @ tinyurl.com/pycon8-cd

Peter Bittner

• Developer (of people, companies, code)
• Co-founder Painless Software
• @peterbittner, django@bittner.it

behave-django

codeship-yaml

sentrylogs

django-apptemplates

djangocms-maps

Hypes are hype!

Any cool stuff here, guys?

... and hype

means crazy.

Our lives, our pain.

Agenda

1. What is CD?
2. Why do it?
3. How do it?
4. Put it into practice
5. Demo, Q&A, workshop

• Hands-on coding (infrastructure)
• Start filling in backlogs (PO)
• Start doing meetings (SM)
• Show how PRs/MRs work (pairing)
• Work with PM + management
• Internal communication,
  discussions (change mgmt)

Continuous Delivery?

   .

As a coach you do:

30%

20%

15%

5%

10%

15%

5%

• (C-level management) hype
• Replacement for Scrum+Kanban?
• 1st principle of Agile Manifesto
• All automatic, just push code!
• TDD, BDD, Selenium, Docker
• Deployment button for business

What Is It?

Definition

... a set of practices and principles ... building, testing, and releasing software, faster and more frequently

... put release schedule in the hands of the business - not IT

Text

Source: https://painless.software/continuous-delivery

It's Agile (Ever Since)

Source: http://agilemanifesto.org/principles.html

TDD, BDD, Selenium

Tests are fundamental for build pipeline.

• No automated verification w/o tests
• No safeguarding against regression
• No safety net against deployments
  that would take the site down

Containers (Docker)

Containers make process easier.

• Feature parity across environments
• Develop, Staging, Production identical
• Simplify deployments, no builds on hosts
• Simplify rollbacks

• On-boarding developers takes days
• "Works on my machine"
• Not releasing enough features
• Only a few can trigger deployment
• "Never deploy on a Friday!"
• Move to a different hoster (cloud?)

Does It Solve My Problems?

 The Definition Said

• Build features faster + more frequently
• Test changes faster + more frequently
• Release changes faster + more frequently
• Everyone (even the client) can release

... and Docker said

• Environment parity (production = staging = dev machine)

Definition: https://painless.software/continuous-delivery

That's the Plan

So, in future, we will:

1. Clone a repo + run a project locally
2. Make, test + push changes
3. Get feedback from build server
4. Have changes reviewed + released

• Version control
• Feature branching - maybe
• Code review, PRs / MRs
• A test runner, task runner
• Build server, CI server, pipeline
• Artifact storage, image registry
• Hosting
• Monitoring

What Do We Need?

Development Process

Someone needs to ship it.

push code

open PR/MR

approve

build

review

feature

production

system

branch

server

system

What to Look at

Someone needs to maintain it.

• Infrastructure as code
• Agent-less deployment
• Stick to standard procedures +
  popular tools (best practices)
• Make it work for everyone

*

Build Servers + Registries

Someone needs to build it.

• Pipeline will build, push and deploy
• Pick a solution or combo you love to work with (GitHub+Travis+DockerHub, GitLab, ...)
• Hosted may be cheaper than self-hosted (TCO)
• Build speed, pleasant UI + workflow may
  pay off more than a smaller price

CI Services: http://alternativeto.net/software/travis-ci/

Registries: http://alternativeto.net/software/docker-hub/

Docker Hosting

Someone needs to serve it.

• Decent Docker hosting supports `docker machine` drivers
• Regular (dedicated) hosting with `docker` and `docker-compose` also works
• Avoid manual setup (transient servers)

Shortlist: https://blog.codeship.com/the-shortlist-of-docker-hosting/

Monitoring

Someone needs to babysit it.

• Screen resource utilization after deployment
• Some issues are only discovered in production
• Capture stack traces, identify bottlenecks
• Get user feedback, communicate w/ users
• Collect historical data

Monitoring: http://alternativeto.net/software/sentry/

Transparency: https://painless.software/transparency

Pythonic Choices

• Tox w/ pytest, Flake8, Pylint, behave
• uWSGI, Nginx, Compose
• README.rst, no docs
• Unit and BDD tests prepared
• GitHub+Travis CI / GitLab / Bitbucket

.travis.yml

language: python
python: 3.5
env:
  matrix:
    - TOXENV=flake8
    - TOXENV=py27
    - TOXENV=py35
install: pip install tox
script: tox
deploy:
  # magic stuff here

tox.ini

[tox]
envlist = flake8,py27,py35
skipsdist = true

[flake8]
exclude = .cache,.git,.tox,build
max-line-length = 80

[pytest]
addopts = --junitxml=unittests.xml
          --strict

docker-compose.yml

version: "2"
services:
  webserver:
    build: ./config/webserver
  application:
    build:
      context: .
      dockerfile: ./config/application/Dockerfile
    user: application
  database:
    image: postgres

uwsgi.ini

[uwsgi]
cheaper = 2
chmod-socket = 660
chown-socket = application:application
enable-threads = True
master = True
processes = 16
python-autoreload = 1
socket = /run/project/uwsgi.sock
wsgi-file =
    /opt/project/application/wsgi.py

nginx.conf

server {
    location / {
        include uwsgi_params;
        uwsgi_pass
            unix:///run/project/uwsgi.sock;
    }
    location /media {
        root /opt/project
    }
    location /static {
        root /opt/project;
    }
}

README.rst

Project Name
============
Awesome demo website for PyCon 8.

Getting Started
---------------
To start developing on this project simply bring up the Docker setup:

.. code-block:: bash

    docker-compose up --build -d
    docker-compose logs -f

Don't Create Those Files!

1. Create a project skeleton (cookiecutter)
2. Use existing cookiecutters
3. Contribute to the one you love best!
4. Do the hard stuff in public (it will pay off)

Grandpa nailed it,
once again.

Just awesome.

I'll tell our developers.

Demo Time!

https://github.com/painless-software/painless-continuous-delivery

https://painless.cloud