Modern Deployment
with
Vagrant, Ansible & Supervisor
About me:
Software Engineer @ Commission Junction
current focus on build automation & release process
Note: view slides depth first!
Deployment Problems?
This is your app in development:
This is your app in production:
What causes service outages?
unreachable devices
unreachable services
limited resources
software bugs
poor configuration
cosmic misalignment
Things we can't control, but also...
things we can control
What about the things we can control?
Question:
How do we fix it?
Answer:
Smaller, more frequent deployments
Deployment should be risk free
Requirements of
Modern Deployment
Ability to deploy services of any shape
Not all software is the same
different software stacks, different hardware requirements, different deployments
Fully automated (single step process)
the devops trinity
automation in tests, deployment, and recovery
Repeatable Process
Independent of time or environment
Should be able to deploy over and over with the same results
Consistent
minimal gotchyas, should work the same, no matter what you need it for
Fully documented
clear & unambiguous
No difference between actual and documented process
Should enable experimentation
patches, rollbacks, subsets
experimental service coordination & deployment should be safe
Which tools to use?
How can we solve our problems while meeting our requirements?
What do we require from our tooling?
mature feature sets & design
active community (not dead)
rich documentation
it should "just work"
Vagrant
Virtual Machine Automation
Life in a box has never been easier
Why Vagrant?
Create and configure lightweight, reproducible, and portable development environments.
lightweight: should be cheap to create and destroy
makes it easy to experiment, build and trash servers as your workflow sees fit
reproducible: what worked once should work again
the process should be repeatable across different development environments
portable: able to be packaged and shipped
system configuration as history, serialize and share
Vagrant is a sandbox for exploring what it takes to configure an environment
Vagrant Configuration
ruby configs, mostly assigning names to values
Configure your cluster
One box or many? Do you want to use any plugins?
Configure your VM Provider
virtualbox, linux containers, vmware
Configure your provisioner
sh scripts, puppet, chef, ansible
Whirlwind tour:
$ vagrant init
create a new vagrant project in the current directory
$ vagrant up
start the vm defined by the local configuration
$ vagrant provision
runs the defined provisioner to configure the local vm
$ vagrant ssh
ssh into the vm managed by vagrant
$ vagrant halt
shuts down a running virtual machine
$ vagrant destroy
deletes the guest machine from the host
Synchronize Your Environment
one configuration process, many environments
Ansible
System Automation over SSH
Radically Simple Orchestration
Why Ansible?
makes your applications and systems easier to deploy.
Consistent & Reliable
Let’s you know what failed, where, and when
Powerful but Simple
compose existing technology
Batteries Included
Common functionality out of the box
Ansible allows simple, single pass configuration
make your machines look the way you need them.
Ansible Architecture
commands: ship an executable module to a remote machine & invoke it with parameters
playbooks: provide execution context for series of composed modules
Anatomy of a Playbook
A playbook is a simple list of plays
A play is a declarative data structure
-
hosts: where it is acceptable for these steps to run?
- vars: do we have any variable context for these tasks?
- tasks: what series of commands define the deployment?
- roles: re-usable and independent collections of tasks & vars
- hosts: web
vars:
port: 8080
tasks:
- name: start the web server
command: /bin/runserver -p {{ port }}
Ansible Inventories
Ansible maps functionality to servers
Inventories map servers to hostgroups
[web]
web[01:10].example.com
[web:vars]
database=db-master.example.com
admin=admin@example.com
Above we define the group 'web' containing 10 servers.
Inventory variables provide information about the host servers
Whirlwind tour
ansible & ansible-playbook
# ansible <hostgroup> <args>
$ ansible web -u webuser -a whoami
=> on all ten web servers... webuser
$ cat playbook.yaml
- hosts: web
remote_user: webuser
tasks:
- command: whoami
# ansible-playbook <playbooks...> <args>
$ ansible-playbook playbook.yaml
=> on all ten web servers... webuser
Who is using ansible?
-
commission junction - deployment automation
-
evernote - deployment automation & config mgmt
-
google - automation of googles elastic compute cloud
-
netflix - deeply integrated with the netflix OSS stack
-
rackspace - automation of creation and teardown of vms
-
amazon - contributor to ansible AWS modules
Of course commission junction was an early adopter (pre 1.0)...
Cowsay Integration
one of the few apps with cowsay integration built in
_________________________________________
/ Ansible is now officially easier for me \ | than writing a shell script to manage a | | server. Casual devops folks should | \ absolutely learn it. / ----------------------------------------- \ ^__^ \ (oo)\_______ (__)\ )\/\ ||----w | || ||
Orchestrate your Environment
You can't do it alone
Supervisor
Service automation
Hassle-free service definitions
Why Supervisor?
Supervisor is a client/server system that allows its users to control a number of processes on UNIX-like operating systems.
Convenience - Simple language for service definition
Accuracy - Know for sure whether an app is running
Delegation - Detach processes from system configuration
Centralized - Single authority, decentralized clients
Supervisor provides the full power of init.d in userland
Supervisor Configuration
One or more INI files
Server
How can you interact with the supervisor server?
(unix socket? http? xml-rpc? auth?)
Program
What does it take to start your service?
(environment? command? directory? user? umask?)
An example config
conf.d/web-server.ini
[program:web-server]
command=python -m SimpleHTTPServer
directory=/home/web/public
What does this get you?
-
query & control
- autostart with supervisor
-
local & remote
- tailing of stdout/err
Compare to shell init...
complicated, repetitive
case "$1" in
start)
start
RETVAL=$?
;;
stop)
stop
RETVAL=$?
;;
restart|force-reload)
restart
RETVAL=$?
;;
reload)
/usr/bin/supervisorctl $OPTIONS reload
RETVAL=$?
;;
*)
echo $"Usage: $0 {start|stop|status|restart|reload|force-reload|condrestart}"
exit 1
esac
;* (
Whirlwind tour
$ supervisord
# starts the daemon, and all supervised apps
$ supervisorctl status
# lists the status of all supervised programs
$ supervisorctl start|stop|restart|update <app>
# control a process
$ supervisorctl reread
# update service definitions
Hang onto your services
Process management should be easy
Closing thoughts
What do these tools have in common?
Declarative Configuration
Flexible & Extensible
Separation of Concerns
Act as Documentation
Aids in consistency
Problems?
Further Resources
deployment
By Thomas Omans
deployment
service deployment with ansible, vagrant, & supervisor
- 3,883