Continuous Delivery
for Python Developers

PyCon 8, 2017

live slides @ tinyurl.com/pycon8-cd

Peter Bittner

  • Developer (of people, companies, code)
  • Co-founder Painless Software
  • @peterbittner, django@bittner.it

behave-django

codeship-yaml

sentrylogs

django-apptemplates

djangocms-maps

Hypes are hype!

Any cool stuff here, guys?

... and hype

means crazy.

Our lives, our pain.

Agenda

  1. What is CD?
  2. Why do it?
  3. How do it?
  4. Put it into practice
  5. Demo, Q&A, workshop

What

Continuous

Delivery?

is

  • Hands-on coding (infrastructure)
  • Start filling in backlogs (PO)
  • Start doing meetings (SM)
  • Show how PRs/MRs work (pairing)
  • Work with PM + management
  • Internal communication,
    discussions (change mgmt)

Continuous Delivery?

   .

As a coach you do:

30%

20%

15%

5%

10%

15%

5%

  • (C-level management) hype
  • Replacement for Scrum+Kanban?
  • 1st principle of Agile Manifesto
  • All automatic, just push code!
  • TDD, BDD, Selenium, Docker
  • Deployment button for business

What Is It?

Definition

... a set of practices and principles ... building, testing, and releasing software, faster and more frequently

... put release schedule in the hands of the business - not IT

Text

It's Agile (Ever Since)

Satisfy the customer!

TDD, BDD, Selenium

Tests are fundamental for build pipeline.

  • No automated verification w/o tests
  • No safeguarding against regression
  • No safety net against deployments
    that would take the site down

TDD good practice

BDD helpful for acceptance

Selenium / performance

!! risk !!

Containers (Docker)

Containers make process easier.

  • Feature parity across environments
  • Develop, Staging, Production identical
  • Simplify deployments, no builds on hosts
  • Simplify rollbacks

Optional, but makes it easier

Container images built by pipeline

Push complete system

Why

Continuous

Delivery

after all?

  • On-boarding developers takes days
  • "Works on my machine"
  • Not releasing enough features
  • Only a few can trigger deployment
  • "Never deploy on a Friday!"
  • Move to a different hoster (cloud?)

Does It Solve My Problems?

 The Definition Said

  • Build features faster + more frequently
  • Test changes faster + more frequently
  • Release changes faster + more frequently
  • Everyone (even the client) can release

... and Docker said

  • Environment parity (production = staging = dev machine)

That's the Plan

So, in future, we will:

  1. Clone a repo + run a project locally
  2. Make, test + push changes
  3. Get feedback from build server
  4. Have changes reviewed + released

All in a matter of minutes

No fear to break anything

Happy devs + clients

How

Continuous

Delivery

do

we

make

happen?

  • Version control
  • Feature branching - maybe
  • Code review, PRs / MRs
  • A test runner, task runner
  • Build server, CI server, pipeline
  • Artifact storage, image registry
  • Hosting
  • Monitoring

What Do We Need?

Development Process

Someone needs to ship it.

push code

open PR/MR

approve

build

review

feature

production

system

branch

server

system

What to Look at

Someone needs to maintain it.

  • Infrastructure as code
  • Agent-less deployment
  • Stick to standard procedures +
    popular tools (best practices)
  • Make it work for everyone

*

Self-explanatory procedures

Don't break conventions

Make developers love it!

Build Servers + Registries

Someone needs to build it.

  • Pipeline will build, push and deploy
  • Pick a solution or combo you love to work with (GitHub+Travis+DockerHub, GitLab, ...)
  • Hosted may be cheaper than self-hosted (TCO)
  • Build speed, pleasant UI + workflow may
    pay off more than a smaller price

Docker Hosting

Someone needs to serve it.

  • Decent Docker hosting supports `docker machine` drivers
  • Regular (dedicated) hosting with `docker` and `docker-compose` also works
  • Avoid manual setup (transient servers)

Monitoring

Someone needs to babysit it.

  • Screen resource utilization after deployment
  • Some issues are only discovered in production
  • Capture stack traces, identify bottlenecks
  • Get user feedback, communicate w/ users
  • Collect historical data

Now

Continuous

Delivery

let's

do

with

Python!

Pythonic Choices

  • Tox w/ pytest, Flake8, Pylint, behave
  • uWSGI, Nginx, Compose
  • README.rst, no docs
  • Unit and BDD tests prepared
  • GitHub+Travis CI / GitLab / Bitbucket

Run project: `docker-compose up`

`manage.py runserver` works too

Run tests: `tox` or `pytest`

.travis.yml

language: python
python: 3.5
env:
  matrix:
    - TOXENV=flake8
    - TOXENV=py27
    - TOXENV=py35
install: pip install tox
script: tox
deploy:
  # magic stuff here

tox.ini

[tox]
envlist = flake8,py27,py35
skipsdist = true

[flake8]
exclude = .cache,.git,.tox,build
max-line-length = 80

[pytest]
addopts = --junitxml=unittests.xml
          --strict

docker-compose.yml

version: "2"
services:
  webserver:
    build: ./config/webserver
  application:
    build:
      context: .
      dockerfile: ./config/application/Dockerfile
    user: application
  database:
    image: postgres

uwsgi.ini

[uwsgi]
cheaper = 2
chmod-socket = 660
chown-socket = application:application
enable-threads = True
master = True
processes = 16
python-autoreload = 1
socket = /run/project/uwsgi.sock
wsgi-file =
    /opt/project/application/wsgi.py

nginx.conf

server {
    location / {
        include uwsgi_params;
        uwsgi_pass
            unix:///run/project/uwsgi.sock;
    }
    location /media {
        root /opt/project
    }
    location /static {
        root /opt/project;
    }
}

README.rst

Project Name
============
Awesome demo website for PyCon 8.

Getting Started
---------------
To start developing on this project simply bring up the Docker setup:

.. code-block:: bash

    docker-compose up --build -d
    docker-compose logs -f

Don't Create Those Files!

  1. Create a project skeleton (cookiecutter)
  2. Use existing cookiecutters
  3. Contribute to the one you love best!
  4. Do the hard stuff in public (it will pay off)

Run `cookiecutter

gh:painless-software/

painless-continuous-delivery`

Then start working immediately!

Grandpa nailed it,
once again.

Just awesome.

I'll tell our developers.

One More Thing ...

Demo Time!

Thank you!

for your precious time

Painless Software

Less pain, more fun.

PyCon 8: Continuous Delivery for Python Developers

By Peter Bittner

PyCon 8: Continuous Delivery for Python Developers

Continuous Delivery sounds easy in theory, but it’s hard to do in practice. There are myriads of things you can and should do to get your code delivered faster, reliably. We look at what we can do as Python developers, or as a small or mid-sized team to make the industrialized software development production chain come true. https://www.youtube.com/watch?v=jgNIrBWB6dM

  • 3,572