Introduction to Ansible

Alejandro Guirao Rodríguez

github.com/lekum

Architecture overview

Source: sysadmincasts

Environment installation

Follow the instructions at

http://github.com/lekum/introduction-to-ansible

Vagrantfile

VAGRANTFILE_API_VERSION = "2"
Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
    config.ssh.insert_key = false
    config.vm.provider :virtualbox do |vb|
        vb.customize ["modifyvm", :id, "--memory", "256"]
    end

    # Web server 1
    config.vm.define "web01" do |web|
        web.vm.hostname = "web01"
        web.vm.box = "ubuntu/trusty64"
        web.vm.network :private_network, ip: "10.0.15.21"
    end

    # Web server 2
    config.vm.define "web02" do |web|
        web.vm.hostname = "web02"
        web.vm.box = "ubuntu/trusty64"
        web.vm.network :private_network, ip: "10.0.15.22"
    end

    # Database server.
    config.vm.define "db01" do |db|
        db.vm.hostname = "db1"
        db.vm.box = "ubuntu/trusty64"
        db.vm.network :private_network, ip: "10.0.15.23"

        config.vm.provider "virtualbox" do |vb|
            vb.customize ['createhd', '--filename', 'disk.vdi', '--size', 5 * 1024]
            vb.customize ['storageattach', :id, '--storagectl', 'SATAController', '--port', 1, '--device', 0, '--type', 'hdd', '--medium', 'disk.vdi']
        end

    end
end

Host file (inventory)

[web]
10.0.15.21
10.0.15.22

[db]
10.0.15.23

# Variables that will be applied to all servers
[all:vars]
ansible_ssh_user=vagrant

More info and options: docs.ansible.com/intro_inventory.html

Ad Hoc Commands

• Throw-away, one-time actions
• Execution of an Ansible module in each host of a group
• Run by the "ansible" command using this syntax:

ansible group -m module -a options -i host_file

Let's assume that the host file is located at /etc/ansible/hosts

Hello World!: Ping

(ansible)alex ~ $ ansible all -m ping
10.0.15.22 | success >> {
    "changed": false, 
    "ping": "pong"
}

10.0.15.21 | success >> {
    "changed": false, 
    "ping": "pong"
}

10.0.15.23 | success >> {
    "changed": false, 
    "ping": "pong"
}

ansible all -m ping

Let's check the hostnames

(ansible)alex ~ $ ansible all -a "cat /etc/hostname"
10.0.15.22 | success | rc=0 >>
web02

10.0.15.21 | success | rc=0 >>
web01

10.0.15.23 | success | rc=0 >>
db1

ansible all -a "cat /etc/hostname"

If no module is specified, then the command module is used

Let's install ntp, first in the db

(ansible)alex ~ $ ansible db -s -m apt -a "name=ntp update_cache=yes"
10.0.15.23 | success >> {
    "changed": true, 
    "stderr": "", 
    "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nThe following extra packages will be installed:\n  libopts25\nSuggested packages:\n  ntp-doc\nThe following NEW packages will be installed:\n  libopts25 ntp\n0 upgraded, 2 newly installed, 0 to remove and 62 not upgraded.\nNeed to get 473 kB of archives.\nAfter this operation, 1676 kB of additional disk space will be used.\nGet:1 http://archive.ubuntu.com/ubuntu/ trusty/main libopts25 amd64 1:5.18-2ubuntu2 [55.3 kB]\nGet:2 http://archive.ubuntu.com/ubuntu/ trusty-updates/main ntp amd64 1:4.2.6.p5+dfsg-3ubuntu2.14.04.2 [418 kB]\nFetched 473 kB in 5s (82.2 kB/s)\nSelecting previously unselected package libopts25:amd64.\n(Reading database ... 60960 files and directories currently installed.)\nPreparing to unpack .../libopts25_1%3a5.18-2ubuntu2_amd64.deb ...\nUnpacking libopts25:amd64 (1:5.18-2ubuntu2) ...\nSelecting previously unselected package ntp.\nPreparing to unpack .../ntp_1%3a4.2.6.p5+dfsg-3ubuntu2.14.04.2_amd64.deb ...\nUnpacking ntp (1:4.2.6.p5+dfsg-3ubuntu2.14.04.2) ...\nProcessing triggers for man-db (2.6.7.1-1ubuntu1) ...\nProcessing triggers for ureadahead (0.100.0-16) ...\nSetting up libopts25:amd64 (1:5.18-2ubuntu2) ...\nSetting up ntp (1:4.2.6.p5+dfsg-3ubuntu2.14.04.2) ...\n * Starting NTP server ntpd\n   ...done.\nProcessing triggers for libc-bin (2.19-0ubuntu6.5) ...\nProcessing triggers for ureadahead (0.100.0-16) ...\n"
}

ansible db -s -m apt -a "name=ntp update_cache=yes"

The -s option runs the module with sudo

Let's restart the ntp service

(ansible)alex ~ $ ansible all -s -m service -a "name=ntp state=restarted"
10.0.15.23 | success >> {
    "changed": true, 
    "name": "ntp", 
    "state": "started"
}

10.0.15.21 | success >> {
    "changed": true, 
    "name": "ntp", 
    "state": "started"
}

10.0.15.22 | success >> {
    "changed": true, 
    "name": "ntp", 
    "state": "started"
}

ansible all -s -m service -a "name=ntp state=restarted"

Enough! No more one-liners!

Ad-hoc commands may be useful for quick-and-dirty operations. But configuration management needs a more disciplined way of working

The playbooks are YAML files that specify a list of plays.

Each play is a series of tasks applied to a group of hosts

Playbooks are played with the ansible-playbook command

A playbook to deploy nginx

---
- hosts: web
  sudo: yes
  vars:
      - external_port: 80
      - internal_port: 8000

  tasks:

      - name: Add the apt repository for nginx
        apt_repository: repo="ppa:nginx/stable" update_cache=yes

      - name: Install nginx
        apt: name=nginx state=present

      - name: Ensure that nginx is started
        service: name=nginx state=started

      - name: Remove default site
        file: path=/etc/nginx/sites-enabled/default state=absent
        notify:
            - Restart nginx

      - name: Configure a site
        template: src=templates/site.j2 dest=/etc/nginx/sites-available/site

      - name: Enable a site
        file: src=/etc/nginx/sites-available/site dest=/etc/nginx/sites-enabled/site state=link
        notify:
            - Restart nginx

  handlers:
      - name: Restart nginx
        service: name=nginx state=restarted

deploy-nginx.yml

Let's run the playbook

$ ansible-playbook deploy-nginx.yml 

PLAY [web] ******************************************************************** 

GATHERING FACTS *************************************************************** 
ok: [10.0.15.21]
ok: [10.0.15.22]

TASK: [Add the apt repository for nginx] ************************************** 
changed: [10.0.15.21]
changed: [10.0.15.22]

TASK: [Install nginx] ********************************************************* 
changed: [10.0.15.22]
changed: [10.0.15.21]

TASK: [Ensure that nginx is started] ****************************************** 
ok: [10.0.15.21]
ok: [10.0.15.22]

TASK: [Remove default site] *************************************************** 
changed: [10.0.15.21]
changed: [10.0.15.22]

TASK: [Configure a site] ****************************************************** 
changed: [10.0.15.21]
changed: [10.0.15.22]

TASK: [Enable a site] ********************************************************* 
changed: [10.0.15.21]
changed: [10.0.15.22]

TASK: [Remove default site] *************************************************** 
ok: [10.0.15.21]
ok: [10.0.15.22]

NOTIFIED: [Restart nginx] ***************************************************** 
changed: [10.0.15.21]
changed: [10.0.15.22]

PLAY RECAP ******************************************************************** 
10.0.15.21                 : ok=9    changed=6    unreachable=0    failed=0   
10.0.15.22                 : ok=9    changed=6    unreachable=0    failed=0  

Encapsulation: roles

In order to create reusable components, we can define roles

Roles are a way to relate certain elements to a group of servers:

• Variables
• Tasks
• Handlers
• Files and templates
• Dependencies to other roles

Let's create a role to install MongoDB

$ tree roles
roles
└── mongodb_server
    ├── defaults
    │   └── main.yml
    ├── handlers
    │   └── main.yml
    ├── README.md
    └── tasks
        └── main.yml

Tasks

---
- name: Add the apt-key for mongodb-org
  apt_key: keyserver=hkp://keyserver.ubuntu.com:80 id=0x7F0CEB10
  sudo: yes

- name: Add the apt repository for mongodb-org
  apt_repository: repo="deb http://downloads-distro.mongodb.org/repo/ubuntu-upstart dist 10gen" update_cache=yes
  sudo: yes

- name: Make sure mongodb-org is installed
  apt: name="mongodb-org" state=latest
  sudo: yes

- name: Enable small journal files if applicable
  lineinfile: dest=/etc/mongod.conf regexp="^smallfiles =" line="smallfiles = true"
  when: small_files
  sudo: yes
  notify:
      - Restart mongodb-org

- name: Bind mongodb to every IP
  lineinfile: dest=/etc/mongod.conf regexp="^bind_ip =" state=absent
  sudo: yes
  notify:
      - Restart mongodb-org

- name: Make sure mongod is started
  service: name=mongod state=started
  sudo: yes

roles/mongodb_server/tasks/main.yml

Handlers

---
- name: Restart mongodb-org
  service: name=mongod state=restarted
  sudo: yes

Defaults

---
small_files: False

roles/mongodb_server/handlers/main.yml

roles/mongodb_server/defaults/main.yml

Using roles in a playbook

---
- hosts: db
  sudo: yes
  roles:
      - mongodb_server

  tasks:

      - name: Install the pre-requisites
        apt: name={{ item }} update_cache=yes
        with_items:
            - lvm2

      - name: Create a vg named vgdata with /dev/sdb
        lvg: vg=vgdata pvs=/dev/sdb

      - name: Create a lv named lvdata01 in vgdata
        lvol: vg=vgdata lv=lvdata01 size=80%VG

      - name: Create an ext4 filesystem in /dev/mapper/vgdata-lvdata01
        filesystem: fstype=ext4 dev=/dev/mapper/vgdata-lvdata01

      - name: Make sure mongod is stopped
        service: name=mongod state=stopped

      - name: Mount the directory
        mount: name=/var/lib/mongodb src=/dev/mapper/vgdata-lvdata01 fstype=ext4 state=mounted

      - name: Re-establish permissions for the directory
        file: path=/var/lib/mongodb owner=mongodb group=nogroup state=directory recurse=yes

      - name: Make sure mongod is started
        service: name=mongod state=started

deploy-mongo.yml

Install MongoDB!

$ ansible-playbook deploy-mongo.yml 

PLAY [db] ********************************************************************* 

GATHERING FACTS *************************************************************** 
ok: [10.0.15.23]

TASK: [mongodb_server | Add the apt-key for mongodb-org] ********************** 
ok: [10.0.15.23]

TASK: [mongodb_server | Add the apt repository for mongodb-org] *************** 
ok: [10.0.15.23]

TASK: [mongodb_server | Make sure mongodb-org is installed] ******************* 
changed: [10.0.15.23]

TASK: [mongodb_server | Enable small journal files if applicable] ************* 
skipping: [10.0.15.23]

TASK: [mongodb_server | Bind mongodb to every IP] ***************************** 
changed: [10.0.15.23]

TASK: [mongodb_server | Make sure mongod is started] ************************** 
ok: [10.0.15.23]

TASK: [Install the pre-requisites] ******************************************** 
changed: [10.0.15.23] => (item=lvm2)

TASK: [Create a vg named vgdata with /dev/sdb] ******************************** 
changed: [10.0.15.23]

TASK: [Create a lv named lvdata01 in vgdata] ********************************** 
changed: [10.0.15.23]

TASK: [Create an ext4 filesystem in /dev/mapper/vgdata-lvdata01] ************** 
changed: [10.0.15.23]

TASK: [Make sure mongod is stopped] ******************************************* 
changed: [10.0.15.23]

TASK: [Mount the directory] *************************************************** 
changed: [10.0.15.23]

TASK: [Re-establish permissions for the directory] **************************** 
changed: [10.0.15.23]

TASK: [Make sure mongod is started] ******************************************* 
changed: [10.0.15.23]

NOTIFIED: [mongodb_server | Restart mongodb-org] ****************************** 
changed: [10.0.15.23]

PLAY RECAP ******************************************************************** 
10.0.15.23                 : ok=15   changed=11   unreachable=0    failed=0   

Fun with AWS

There are many modules to manage AWS:

• Launch and terminate EC2 instances
• Manage EIP
• S3 objects upload / download
• ELB management
• RDS
• Route 53
• Elasticache...

In order to use them, you will need to:

export AWS_ACCESS_KEY_ID='AK123'
export AWS_SECRET_ACCESS_KEY='abc123'

pip install boto

Let's create an EC2 instance

We will create the instance using the ec2 module

Once it is created, we register the result of the module in a variable so we can access the data of the instance.

We then use the add_host module to dinamically add the instance to the inventory...

... so that we can provision it in the next play of the playbook!

# Example to create an AWS instance and install MongoDB
#
# The region, key_name, image and security group (group) must match your configuration
#
# Before running the playbook, ensure that:
#  - The key 'demo-key' has been added to ssh-agent
#  - export ANSIBLE_HOST_KEY_CHECKING=False
#  - The environment variables AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY match your credentials
#
---
- hosts: localhost
  connection: local
  gather_facts: no
  tasks:

      - name: EC2 provisioning of MongoDB instance
        ec2:
            region: eu-west-1
            key_name: demo-key
            instance_type: t2.micro
            image: ami-234ecc54
            wait: yes
            instance_tags:
                Name: mongo-server
            exact_count: 1
            count_tag:
                Name: mongo-server
            group:
                - SSH-ACCESS
            volumes:
                - device_name: /dev/xvdc
                  # /dev/xvdc of 5 GB
                  volume_size: 5
        register: ec2

      - name: Wait for SSH to come up
        wait_for: host={{ item.public_ip }} port=22 delay=10 timeout=320 state=started
        with_items: ec2.instances

      - name: Add the host to the group to be provisioned
        add_host: name={{ item.public_ip }} groupname=to_be_provisioned
        with_items: ec2.instances

- hosts: to_be_provisioned
  user: ubuntu
  sudo: yes
  roles:

      - {
          role: mongodb_server,
          small_files: true
        }

  tasks:

      - name: Install the pre-requisites
        apt: name={{ item }} update_cache=yes
        with_items:
            - lvm2

      - name: Create a vg named vgdata with /dev/xvdc
        lvg: vg=vgdata pvs=/dev/xvdc

      - name: Create a lv named lvdata01 in vgdata
        lvol: vg=vgdata lv=lvdata01 size=80%VG

      - name: Create an ext4 filesystem in /dev/mapper/vgdata-lvdata01
        filesystem: fstype=ext4 dev=/dev/mapper/vgdata-lvdata01

      - name: Make sure mongod is stopped
        service: name=mongod state=stopped

      - name: Mount the directory
        mount: name=/var/lib/mongodb src=/dev/mapper/vgdata-lvdata01 fstype=ext4 state=mounted

      - name: Re-establish permissions for the directory
        file: path=/var/lib/mongodb owner=mongodb group=nogroup state=directory recurse=yes

      - name: Make sure mongod is started
        service: name=mongod state=started

Here we go!

deploy-aws-mongo.yml

Managing dynamic inventory

Hardcoding IPs in a file is not a valid way to manage a cloud. We need a dynamic inventory

Ansible provides dynamic inventory scripts for many clouds. The files ec2.py and ec2.ini for AWS are in github.com/ansible/ansible/tree/devel/plugins/inventory

You can inspect your cloud just typing:

./ec2.py --list

The dynamic inventory script creates groups based on tags (super cool!)

Rolling updates

Using pre_tasks and post_tasks, and modules to manage load balancers, we can create a playbook to perform a rolling update

The variable serial controls the number of servers that are concurrently updated

Rolling update in AWS

---
- hosts: web
  serial: 3
  pre_tasks:

      - name: Gathering ec2 facts
        action: ec2_facts

      - name: Instance de-register
        local_action:
            module: ec2_elb
            instance_id: "{{ ansible_ec2_instance_id }}"
            state: 'absent'
  roles:

      - role_that_updates

  post_tasks:

      - name: Instance register
        local_action:
            module: ec2_elb
            instance_id: "{{ ansible_ec2_instance_id }}"
            ec2_elbs: "{{ item }}"
            state: 'present'
        with_items: ec2_elbs

ansible-playbook -i ec2.py rolling-update.yml

rolling-update.yml

More advanced topics

• Complex templating, filters and variable manipulation
• Asynchronous actions
• Local playbooks and delegation
• Ansible pull mode
• Using ansible-vault to encrypt data
• Development of new modules
• ...

Further reading

Great documentation at docs.ansible.com

Examples at github.com/ansible/ansible-examples

Good books:

Ansible for DevOps

Ansible for AWS

Ansible Up & Running

Ansible Weekly Newsletter

Happy hacking!

github.com/lekum