Fabric: python library for devops tasks

Petr Nuzhnov, Five9

Why we need common system configuration managment (devops) tools?

stage #1 "God Mode"

stage #2 "Hell's tooling"

stage #3 'LITTLE CRASH'

Fabric!

Python library and command-line tool for streamining the use of SSH for application deployment or systems administration tasks

Jeff Forcier

@bitprophet

  • paramiko
  • pyinvoke

Author

Installation

pip install fabric

requires:

  • ecdsa
  • paramiko
  • pycrypto

Example

#fabfile.py

from fabric.api import run

def list_directory():
    run('ls -la')
MacBook-Pro-Petr:fabr petrnuzhnov$ fab -H hostname.server.com list_directory -u username -p password
[hostname.server.com] Executing task 'list_directory'
[hostname.server.com] run: ls -la
[hostname.server.com] out: total 483168
[hostname.server.com] out: drwxr-xr-x. 41 username username      4096 Jun 16 07:22 .
[hostname.server.com] out: drwxr-xr-x.  4 root     root          4096 Dec 18  2014 ..
[hostname.server.com] out: -rw-------   1 username username     50498 Jun 16 10:00 .bash_history
[hostname.server.com] out: -rw-r--r--.  1 username username        18 Oct 16  2014 .bash_logout
[hostname.server.com] out: -rw-r--r--   1 username username       269 Jul 15  2015 .bash_profile
[hostname.server.com] out: -rw-r--r--   1 username username       174 Apr 15  2015 .bashrc
[hostname.server.com] out: drwxr-xr-x   2 username username      4096 Feb 27  2015 bin
[hostname.server.com] out: drwxr-xr-x   2 username username      4096 Oct  5  2015 Builds
[hostname.server.com] out: -rw-r--r--   1 username username    118135 May 11 03:08 temp.my
[hostname.server.com] out: drwxr-xr-x   3 username username      4096 May 10 03:25 .vim
[hostname.server.com] out: -rw-------   1 username username     13406 Jun 15 13:32 .viminfo
[hostname.server.com] out: -rw-r--r--   1 username username        90 Oct 29  2014 .vimrc

Main components

  • fab - command line utility
  • fabfile - file with described tasks
  • env - singleton dict subclass, which contains all necessary data: hosts, roles, credentials and other userfull settings
  • task - separate execution unit (e.g. function)

 

fab options and arguments

  • -f  - execution file
  • -H  - list of hosts
  • first_task second_task - tasks which will be executed one by one
  • - u and -p - credentials

task:value passing parameters to your task

fab -f my_task_list.py -H hosts first_task second_task:param1, param2 -u user -p passw

Env

fabric.static.env

Configuration dict which is shared between tasks

  • env.hosts - list of hosts which will be involved
  • env.user
  • env.password
  • env.key_filename - path to ssh private key path
  • env.timeout - default timeout
  • env.keepalive=1 (default=0)

...

 

Hosts definition

env.roledefs['web'] = ['host1', 'host2',..]
env.roledefs['db'] = ['host3', 'host4']
env.roledefs = {'web':{'hosts':['host1', 'host2',..],
                       'logs':'/var/log/django'}

                'db':{'hosts':['host2', 'host3',...],
                      'logs':'/var/log/mysql'}}
env.hosts = ['host1', 'host2', ...]

'username@hostname:port'

OR

SETTINGS

Simple env wrapper

def task():
    with settings(warn_only=True):
        #do something dangerous
def task():
    with settings(timeout=30):
        sudo('service mysqld restart')
def task():
    with settings(sudo_user=petrnuzhnov):
        run('cat ~/my_personal_file.txt')

Tasks defenition

# fabfile.py
def task1():
    """Task1 docstring"""
    pass

def task2():
    """Task2 docstring"""
    pass

def task_combined():
    """Task_combined docstring"""
    task1()
    task2()

def _not_a_task()
    print "I'm not a task function"
fab --list
Available commands:
    task1            Task1 docstring
    task2            Task2 docstring
    combined_task    Task_combined docstring

Task definition #2

# fabfile.py
from fabric.api import task, roles, run, env

@roles('web')
@task()
def stop_web():
    sudo('service nginx stop')

@roles('db')
@task()
def stop_db():
    sudo('service mysqld stop')

@task(default=True)
def stop_all():
    execute(stop_web)
    execute(stop_db)

def not_a_task()
    print "I'm not a task function"
fab --list
Available commands:

    stop_web
    stop_db
    stop_all

BUILT-In Operations

  • run
  • local
  • open_shell
  • require
  • reboot
  • prompt
  • sudo
  • cd
  • lcd
  • get
  • put

Execution (parallel and serial) on hosts

fab -H host1, host2, host3, host4, host5 parallel_task serial_task

@task
@parallel(pool_size=5)
def parallel_task():
    # task will be started on hosts in parallel
    ...

@task
def serial_task():
    # will be in serial
    ...
fab -H host1, host2, host3, host4, host5 -P -z 5 parallel_task serial_task

# fabfile.py
...

@task
def parallel_task():
    # task will be started on hosts in parallel
    ...

@task
@serial
def serial_task():
    # will be in serial
    ...

OR

Pros

  • python-user-friendly
  • no client side
  • flexible
  • easy to understand what is going on by reading of script

Cons

  • python 2.x
  • no parallel tasks under windows
  • not a silver bullet

Thank you!

Questions ?

fabric: python library for devops tasks

By Nuzhnov Petr

fabric: python library for devops tasks

Presentation for NN python meetup rannts#11

  • 1,098