Easy FullStack Deployments

Who I am?

  • Alvaro Aguirre
  • ALMA Project
  • Co-founder PythonChile

ALMA Observatory

Atacama Large Millimeter/submillimeter Array

66 antennas

ways to build software

Blackbox deployments

Separation of Concerns

Docker

Ansible

Buildout

App

Buildout

Buildout is a Python-based build system for creating, assembling and deploying applications from multiple parts, some of which may be non-Python-based. It lets you create a buildout configuration and reproduce the same software later.

Ansible

Ansible is the simplest way to automate apps and IT infrastructure. Application Deployment + Configuration Management + Continuous Delivery.

Docker

an open platform for developers and sysadmins to build, ship, and run distributed applications

Live Coding!

The training setup

  • Pair programming
  • AWS machine located in Frankfurt
          - m.large
          - 7.5 GB RAM
  • Each 'pair' will access to specific container.
    ​      - ssh root@52.28.125.90 -p <port>
          - Password: 123456
  • You can see the application running at:
          - http://52.28.125.90:<http-port>
  • docker run -d –name training -e ROOT_PASS="123456" –privileged -P -i "aaguirre/ep2015:latest"

www.stacklab.io/ep2015

docker pull aaguirre/ep2015


https://slides.com/aaguirre/deck




home
├── lab01
│   └── env
├── lab02
│   ├── bin
│   ├── develop-eggs
│   ├── eggs
│   ├── parts
│   └── src
├── lab03
│   ├── bin
│   ├── develop-eggs
│   ├── eggs
│   ├── parts
│   └── src
├── lab04
│   ├── ansible
│   ├── app
│   └── env
├── lab05
│   ├── ansible
│   ├── docker -> /etc/offline/docker
│   └── env
└── lab06
    ├── env
    └── my.app

Lab 01

Installing Pyramid

Lab 01

cd /home/lab01/

virtualenv env

source env/bin/activate   (deactivate)

pip install pyramid

pcreate -s starter my.app

cd my.app/

python setup.py develop

pserve development.ini   

http://52.28.125.90:<port>

Lab 01

python setup.py test -q

pcreate -l                (list available templates)
           
pshell development.ini 

Lab 02

Installing Pyramid using Buildout

Lab 02

Buildout

Buildout is a Python-based build system for creating, assembling and deploying applications from multiple parts, some of which may be non-Python-based. It lets you create a buildout configuration and reproduce the same software later.

Lab 02

cd /home/lab02/

           
[buildout]
extends =
    versions.cfg

show-picked-versions = true

versions = versions

eggs-directory = eggs

parts =
    pyramid


[pyramid]
recipe = zc.recipe.egg
dependent-scripts = true
eggs =
    pyramid

interpreter = py

Lab 02

#curl -O https://bootstrap.pypa.io/bootstrap-buildout.py

python bootstrap-buildout.py

bin/buildout -o

ls bin/

Lab 02

Creating a project inside buildout


# bin/pcreate -s starter src/my.app

Lab 02

Creating a project inside buildout

[buildout]
extends =
    versions.cfg

show-picked-versions = true

develop =
    src/my.app

versions = versions

eggs-directory = eggs

parts =
    pyramid
    my.app


[pyramid]
recipe = zc.recipe.egg
dependent-scripts = true
eggs =
    pyramid

interpreter = py

Lab 02

Running buildout


bin/buildout -o

bin/pserve src/my.app/development.ini

http://52.28.125.90:<port>

Lab 03

Using buildout + git (or svn, cvs, etc)

Lab 03

[buildout]
extends =
    versions.cfg

show-picked-versions = true

versions = versions

extensions =
    mr.developer

sources = sources

always-checkout = force

auto-checkout =
    my.app

eggs-directory = eggs

parts =
    app

[sources]
my.app = git https://github.com/aaguirre/ep2015.app1.git

[app]
recipe = zc.recipe.egg
dependent-scripts = true
eggs =
    my.app

interpreter = py
cd /home/lab03/

           

Lab 03


bin/buildout -o

bin/pserve src/my.app/development.ini

Lab 04

Using ansible to deploy

Lab 04

Ansible

Ansible is the simplest way to automate apps and IT infrastructure. Application Deployment + Configuration Management + Continuous Delivery.

Ansible

Ansible Client

Host 1

Host 2

Host 3

Host 4

Host 5

Lab 04

cd /home/lab04/

           

ansible/

 

app/

 

env/          (source env/bin/activate)

Lab 04



├── group_vars
│   └── all
├── hosts
├── play.yml
└── roles
    └── myapp
        ├── files
        │   ├── bootstrap-buildout.py
        │   ├── buildout.cfg
        │   └── versions.cfg
        ├── handlers
        │   └── main.yml
        ├── meta
        │   └── main.yml
        ├── tasks
        │   └── main.yml
        ├── templates
        └── vars
            └── main.yml
cd /home/lab04/

           

Lab 04

- name: start myapp
  command: bin/pserve src/my.app/development.ini --daemon --pid=app.pid
  args:
     chdir: "{{ installation_path }}"
  notify:
    - wait for service

- name: wait for service
  wait_for: port=6543 delay=20
handlers/main.yml

           

 

Lab 04



ansible-playbook -i hosts play.yml
handlers/main.yml

           

 

Lab 04


- name: Copy buildout file to the destination directory
  copy: src=buildout.cfg dest={{ installation_path }}

- name: Copy versions file to the destination directory
  copy: src=versions.cfg dest={{ installation_path }}

- name: Copy boostrap.py file to the destination directory
  copy: src=bootstrap-buildout.py dest={{ installation_path }}

- name: Run Bootstrap
  command: "{{ python_path }}/python bootstrap-buildout.py"
  args:
     chdir: "{{ installation_path }}"

- name: Run buildout
  command: bin/buildout -o
  args:
     chdir: "{{ installation_path }}"
  notify:
     - start myapp
tasks/main.yml
           

pkill pserve

Lab 05

Using Docker (+ ansible)

https://ictwiki.alma.cl/twiki/bin/view/SoftOps/OfflineServicesDeployment#F2f_Workshop_SCO

Lab 05

Docker Architecture

Lab 05

CENTOS 6.6

SSH Server

Buildout

Supervisor

Dockerfile

6543

22

Lab 05

Management

  • Docker CLI
  • Ansible
  • Docker Py

Lab 05

Docker commands

docker build

 

docker pull

 

docker run

Lab 05

Docker commands

docker ps   (list active containers)
 

docker ps -a (list all containers)
 

docker images (list all images)

 

docker rm <container>

 

docker rmi <container>

Lab 05

Lab 05



docker build -t imyapp:latest .


docker run -d -p 6543:6543  imyapp:latest

docker stop <container-id>
cd /home/lab05/docker/

           

Lab 05

pkill docker

rm -rf /var/lib/docker/*

cp -r /etc/offline/dockerdata/* /var/lib/docker/

supervisorctl restart wrapdocker

cd /home/lab05/docker/

           

Ansible

Ansible Client

Container

Docker

Container

Docker

Container

Docker

Container

Docker

Container

Docker

Lab 05

Lab 05

- name: pre
  command: ls
  ignore_errors: yes

- name: run my application image
  docker:
     name: "myapp2"
     state: started
     image: "imyapp:latest"
     ports:
        - "6543:6543"
cd /home/lab05/ansible/

           

Lab 06

Fullstack app

ways to build software

Component architecture

(or service)

Client

Dependency Injection

Interface

Service

Service

Service

from zope.interface import Interface

class INotification(Interface):
    """ Service responsible of sending notifications ."""


    def send(self, message): pass
@subscriber(NewNotification)
@greenlet
def new_notification(event):   
    utilities = event.request.registry.getUtilitiesFor(INotification)
    for name, utility in utilities:
        utility.send(event.message )
config.registry.registerUtility(MailService(settings), INotification, 'mail')

config.registry.registerUtility(Facebook(settings), INotification, 'facebook')

config.registry.registerUtility(Twitter(settings), INotification, 'twitter')

from portal.interfaces import IMailSender

from pyramid_mailer import mailer_factory_from_settings

from pyramid_mailer.message import Message

from zope.interface import implements


class MailService(object):

    implements(IMailSender)

    def __init__(self, settings):
        self.settings = settings
        self.mailer = mailer_factory_from_settings(settings)

    def send(self, message):
        mail = Message(subject=self.settings['subject'],
                       recipients=[self.settings['to']],
                       body=message,
                       sender=self.settings['mail.default_sender'],
                       extra_headers=dict(From=self.settings['mail.from'])
                       )
        self.mailer.send_immediately(mail, fail_silently=False)

docker pull aaguirre/ep2015

 

https://slides.com/aaguirre/deck

 

twitter.com/alvaro_aguirre

 

Thank you!

EuroPython 2015: Easy Full Stack Deployments

By aaguirre

EuroPython 2015: Easy Full Stack Deployments

  • 2,971