Modern Deployment

with 

Vagrant, Ansible & Supervisor 

About me:


Software Engineer @ Commission Junction
current focus on build automation & release process

Note: view slides depth first!




Deployment Problems?

This is your app in development:


This is your app in production:



What causes service outages?

unreachable devices
unreachable services
limited resources
software bugs
poor configuration
cosmic misalignment

Things we can't control, but also...
things we can control


What about the things we can control?


Question:
How do we fix it?

Answer:
Smaller, more frequent deployments
Deployment should be risk free




Requirements of 

Modern Deployment


Ability to deploy services of any shape


Not all software is the same
different software stacks, different hardware requirements, different deployments


Fully automated (single step process)
the devops trinity
automation in tests, deployment, and recovery

Repeatable Process


Independent of time or environment
Should be able to deploy over and over with the same results

Consistent


minimal gotchyas, should work the same, no matter what you need it for

Fully documented


clear & unambiguous
No difference between actual and documented process

Should enable experimentation


patches, rollbacks, subsets
experimental service coordination & deployment should be safe





Which tools to use?

How can we solve our problems while meeting our requirements?


What do we require from our tooling?


mature feature sets & design

active community (not dead)

rich documentation

it should "just work"

Vagrant

Virtual Machine Automation

Life in a box has never been easier

Why Vagrant?

Create and configure lightweight, reproducible, and portable development environments.

lightweight: should be cheap to create and destroy
makes it easy to experiment, build and trash servers as your workflow sees fit
reproducible: what worked once should work again
the process should be repeatable across different development environments
portable: able to be packaged and shipped
system configuration as history, serialize and share

Vagrant is a sandbox for exploring what it takes to configure an environment

Vagrant Configuration

ruby configs, mostly assigning names to values


Configure your cluster
One box or many? Do you want to use any plugins?

Configure your VM Provider
virtualbox, linux containers, vmware

Configure your provisioner
sh scripts, puppet, chef, ansible

Whirlwind tour:

$ vagrant init
create a new vagrant project in the current directory
$ vagrant up
start the vm defined by the local configuration
$ vagrant provision
runs the defined provisioner to configure the local vm
$ vagrant ssh
ssh into the vm managed by vagrant
$ vagrant halt
shuts down a running virtual machine
$ vagrant destroy
deletes the guest machine from the host

Synchronize Your Environment


one configuration process, many environments

Ansible

System Automation over SSH

Radically Simple Orchestration

Why Ansible?

makes your applications and systems easier to deploy.


Consistent & Reliable

Let’s you know what failed, where, and when

Powerful but Simple

compose existing technology

Batteries Included

Common functionality out of the box


Ansible allows simple, single pass configuration 

make your machines look the way you need them.

Ansible Architecture

commands: ship an executable module to a remote machine & invoke it with parameters
playbooks: provide execution context for series of composed modules

Anatomy of a Playbook

A playbook is a simple list of plays
A play is a declarative data structure

  • hosts: where it is acceptable for these steps to run?
  • vars: do we have any variable context for these tasks?
  • tasks: what series of commands define the deployment?
  • roles: re-usable and independent collections of tasks & vars 


 - hosts: web   vars:     port: 8080   tasks:     - name: start the web server       command: /bin/runserver -p {{ port }}

Ansible Inventories

Ansible maps functionality to servers
Inventories map servers to hostgroups 

[web]web[01:10].example.com
[web:vars]database=db-master.example.comadmin=admin@example.com

Above we define the group 'web' containing 10 servers.
Inventory variables provide information about the host servers

Whirlwind tour

ansible & ansible-playbook 

# ansible <hostgroup> <args>$ ansible web -u webuser -a whoami=> on all ten web servers... webuser $ cat playbook.yaml- hosts: web  remote_user: webuser  tasks:    - command: whoami
# ansible-playbook <playbooks...> <args>$ ansible-playbook playbook.yaml=> on all ten web servers... webuser


Who is using ansible?


  • commission junction - deployment automation
  • evernote - deployment automation & config mgmt
  • google - automation of googles elastic compute cloud
  • netflix - deeply integrated with the netflix OSS stack
  • rackspace - automation of creation and teardown of vms
  • amazon - contributor to ansible AWS modules 

Of course commission junction was an early adopter (pre 1.0)...

Cowsay Integration

one of the few apps with cowsay integration built in

  _________________________________________ / Ansible is now officially easier for me \
| than writing a shell script to manage a |
| server. Casual devops folks should      |
\ absolutely learn it.                    /
 ----------------------------------------- 
        \   ^__^
         \  (oo)\_______
            (__)\       )\/\
                ||----w |
                ||     ||


Orchestrate your Environment


You can't do it alone

Supervisor

Service automation


Hassle-free service definitions 

Why Supervisor?


Supervisor is a client/server system that allows its users to control a number of processes on UNIX-like operating systems.


Convenience - Simple language for service definition
Accuracy - Know for sure whether an app is running
Delegation - Detach processes from system configuration
Centralized - Single authority, decentralized clients

Supervisor provides the full power of init.d in userland

Supervisor Configuration

One or more INI files

Server
How can you interact with the supervisor server?
(unix socket? http? xml-rpc? auth?)

Program
What does it take to start your service?
(environment? command? directory? user? umask?)

An example config


conf.d/web-server.ini
[program:web-server]command=python -m SimpleHTTPServerdirectory=/home/web/public

What does this get you?
  • query & control
  • autostart with supervisor
  • local & remote
  • tailing of stdout/err

Compare to shell init...

 complicated, repetitive
case "$1" in
    start)
        start
        RETVAL=$?
        ;;
    stop)
        stop
        RETVAL=$?
        ;;
    restart|force-reload)
        restart
        RETVAL=$?
        ;;
    reload)
        /usr/bin/supervisorctl $OPTIONS reload
        RETVAL=$?
        ;;
    *)
        echo $"Usage: $0 {start|stop|status|restart|reload|force-reload|condrestart}"
        exit 1
esac
 ;* (

Whirlwind tour


$ supervisord# starts the daemon, and all supervised apps
$ supervisorctl status# lists the status of all supervised programs
$ supervisorctl start|stop|restart|update <app># control a process
$ supervisorctl reread# update service definitions

Hang onto your services


Process management should be easy




Closing thoughts

What do these tools have in common?


Declarative Configuration

Flexible & Extensible

Separation of Concerns

Act as Documentation

Aids in consistency

Problems?



Further Resources


Live in-terminal demo

Vagrant
Ansible
Supervisor

@omniegg
Made with Slides.com