ALMA Offline Environment with Docker

 

http://slides.com/aaguirre/deck-1-3

http://slides.com/aaguirre/deck-1-3/live

Agenda

  • Current Situation
  • Docker
  • LAB 1
  • LAB 2

Past

  • ACS 

  • Fat wars

  • ACS libs dependencies

  • Common environment

  • Kickstart approach

  • Redhat 6.6

Offline applications

100 ~

In-house applications

25 ~

Current Situation

...near future

  • Docker in production

  • Docker Registry

  • Continuous integration

  • Development environment

Current Infrastructure

VCenter

Blade

VM

VM

VM

VM

VM

VM

DNS

10.200.117.XX -> hostname

Current Architecture

Apache

VM

VM

VM

VM

VM

VM

Proxy

Tomcat

Database

Archive

NGAS

Docker

an open platform for developers and sysadmins to build, ship, and run distributed applications

...is a platform for:

  • Deploy your applications
  • Package your applications/environments
  • Distribute your applications/environments

Lightweight

 

 

  • share same host kernel
  • more efficient use of RAM
  • layered filesystem (AUFS)

Portability

 

Isolation (cgroups, kernel namespaces, single Linux instances)

 

Performance

 

Modularity

 

Open Source

For us:

 

  • solves differences between development, testing and production environments.
  • distribute applications and environments.
  • faster deployments

Docker Architecture

Current Infrastructure

VCenter

Blade

VM

VM

VM

VM

VM

VM

DNS

10.200.117.XX -> hostname

Docker Infrastructure

Host

Container

Multiple IP interfaces

10.200.117.XX -> hostname

Docker

Container

Docker

Container

Docker

Container

Docker

Container

Docker

Docker

Dockerfiles

FROM rhel6.6:latest

MAINTAINER Alvaro Aguirre <aaguirre@alma.cl>

ENV EPEL_VERSION 6-8
ENV JAVA_VERSION 8u45
ENV BUILD_VERSION b14
ENV TOMCAT_VERSION 7.0.62-4
ENV RHN_USER almasoftware
ENV RHN_PASS XXXXXX


WORKDIR /root

RUN /usr/sbin/subscription-manager register --username=$RHN_USER --password=$RHN_PASS --serverurl=subscription.rhn.redhat.com --autosubscribe

RUN  yum -y update && \
     yum install -y \
     vim-enhanced \
     blas \ 
     blas-devel \
     tetex-latex \
     gcc-c++ \ 
     python-devel \
     python-lxml \
     wget \ 
     openssl \
     openssl-devel \
     zlib-dev 

RUN yum -y install openssh-server  && \
    rm -f /etc/ssh/ssh_host_dsa_key /etc/ssh/ssh_host_rsa_key && \
    ssh-keygen -q -N "" -t dsa -f /etc/ssh/ssh_host_dsa_key && \
    ssh-keygen -q -N "" -t rsa -f /etc/ssh/ssh_host_rsa_key && \
    sed -i "s/#UsePrivilegeSeparation.*/UsePrivilegeSeparation no/g" /etc/ssh/sshd_config && \
    sed -i "s/UsePAM.*/UsePAM no/g" /etc/ssh/sshd_config

ENV AUTHORIZED_KEYS **None**

RUN echo root:'$1$rrii2oPl$sIWEJP2utD9ks3Kn8r.Lu/' | chpasswd --encrypted

RUN wget http://download.fedoraproject.org/pub/epel/6/i386/epel-release-$EPEL_VERSION.noarch.rpm && \
    rpm -ivh epel-release-$EPEL_VERSION.noarch.rpm

RUN yum -y install pyephem \
    python-pip && \
    easy_install -U distribute && \
    pip install numpy==1.7.0

RUN yum -y install freetype-devel \
    libpng-devel && \
    pip install matplotlib

RUN pip install supervisor

RUN wget --no-cookies --no-check-certificate --header "Cookie: oraclelicense=accept-securebackup-cookie" "http://download.oracle.com/otn-pub/java/jdk/$JAVA_VERSION-$BUILD_VERSION/jdk-$JAVA_VERSION-linux-x64.rpm" -O /tmp/jdk-8-linux-x64.rpm

RUN yum -y install /tmp/jdk-8-linux-x64.rpm

COPY apache-tomcat-$TOMCAT_VERSION.el6.x86_64.rpm /root/

RUN yum -y --nogpgcheck localinstall apache-tomcat-$TOMCAT_VERSION.el6.x86_64.rpm

COPY supervisord.conf /etc/supervisord.conf

EXPOSE 8080 22

CMD ["/usr/bin/supervisord"]

https://svn.alma.cl/p2/trunk/ADC/SW/OFFLINE/Docker/offline/

REDHAT 6.6

SSH Server

Java + Tomcat

Supervisor

Docker Layers

8080

22

Supervisor(d)

 is a client/server system that allows its users to monitor and control a number of processes on UNIX-like operating systems.

Management

  • Docker CLI
  • Docker API
  • Ansible

Docker CLI

docker build

 

docker pull

 

docker run

Docker CLI

docker ps   (list active containers)
 

docker ps -a (list all containers)
 

docker images (list all images)

 

docker rm <container>

 

docker rmi <image>

 

docker stats <container>


docker run ioffline:latest

docker run -d ioffline:latest

docker run -d --name protrack ioffline:latest

...defining a name


docker run -d --name protrack -P ioffline:latest

docker run -d --name protrack -p 9000:8080 ioffline:latest

...defining a port


docker run -d --name protrack -p 9000:8080 -v /host/path/:/etc/offline:ro ioffline:latest

...adding a volume


docker run -d --name cas_2015.4 
    -v /etc/offline/:/etc/offline/ 
    -v /etc/offline/webapps/cas/:/var/lib/tomcat/webapps/ 
    -p=10.200.67.22:22:22 
    -p=10.200.67.22:8080:8080 
    --dns=10.200.16.22 
    --hostname=hybris.sco.alma.cl 
    ioffline:latest

docker run -d --name submission_2015.4 
    -v /etc/offline/:/etc/offline/ 
    -v /etc/offline/webapps/submission/:/var/lib/tomcat/webapps/ 
    -v /etc/offline/data/submission/mail.jar:/usr/share/tomcat/lib/mail.jar 
    -v /etc/offline/data/submission/ojdbc6.jar:/usr/share/tomcat/lib/ojdbc6.jar 
    -v /etc/offline/data/submission/xdb.jar:/usr/share/tomcat/lib/xdb.jar 
    -v /etc/offline/data/submission/xmlparserv2.jar:/usr/share/tomcat/lib/xmlparserv2.jar 
    -v /etc/offline/data/submission/server.xml:/etc/tomcat/server.xml 
    -v /etc/offline/data/submission/InvestigatorDbWS.xml:/etc/tomcat/Catalina/localhost/InvestigatorDbWS.xml 
    -v /etc/offline/data/submission/ObsprepSubmissionService.xml:/etc/tomcat/Catalina/localhost/ObsprepSubmissionService.xml 
    -p=10.200.67.34:22:22 
    -p=10.200.67.34:8180:8180 
    --dns=10.200.16.22 
    --hostname=eichsfeldia.sco.alma.cl 
    ioffline:latest 

docker run -d --name scheduling_2015.4 
    -v /etc/offline/:/etc/offline/ 
    -v /etc/offline/webapps/scheduling/:/var/lib/tomcat/webapps/ 
    -p=10.200.67.33:22:22 -p=10.200.67.33:8080:8080 
    --dns=10.200.16.22 
    --hostname=bathilde.sco.alma.cl 
    -e "APRC_WORK_DIR=/eqtc/offline/config/scheduling/work_dir" 
    ioffline:latest

Docker API

 

 

 

 

http://f2f-workshop.sco.alma.cl:4243/images/json

  • Create/Build
  • Run
  • Read Logs
  • Monitor

Ansible

Ansible is the simplest way to automate apps and IT infrastructure. Application Deployment + Configuration Management + Continuous Delivery.

Ansible

Ansible Client

Host 1

Host 2

Host 3

Host 4

Host 5

Ansible

Ansible Client

Container

Docker

Container

Docker

Container

Docker

Container

Docker

Container

Docker

.
├── group_vars
│   └── all
├── hosts
├── play.yml
└── roles
    └── cas
        ├── files
        ├── handlers
        │   └── main.yml
        ├── meta
        │   └── main.yml
        ├── tasks
        │   └── main.yml
        ├── templates
        └── vars
            └── main.yml

Ansible Project

- name: create new virtual ip in the docker host
  command: ip addr add {{cas_ip}}/24 dev eth0
  ignore_errors: yes

- name: run cas application image
  docker:
     name: "cas_{{ release }}"
     state: started
     image: "offline:latest"
     volumes:
        - /etc/offline/:/etc/offline/
        - /etc/offline/webapps/cas/:/var/lib/tomcat/webapps/
     ports:
        - "{{ cas_ip }}:22:22"
        - "{{ cas_ip }}:8080:8080"
     dns:
        - "10.200.16.22"
     hostname: "{{ cas_hostname }}"

Ansible Project

ansible-playbook -i hosts play.yml

...near future

  • Docker in production

  • Docker Registry

  • Continuous deployment/integration

  • Development environment

Lab 01

Installing an OFFline environment

Lab 01

https://ictwiki.alma.cl/twiki/bin/view/SoftOps/OfflineServicesDeployment#F2f_Workshop_SCO

Lab 01

name.sco.alma.cl     10.200.67.XX

user: root

pass: 123456

 

Ports: 22, 8080

 

Proxy:

           f2f-XX.asa-test.alma.cl/protrack

           f2f-XX.asa-test.alma.cl/sc

Lab 01

https://svn.alma.cl/p2/trunk/ADC/SW/OFFLINE/Docker/offline/

Offline Dockerfile

Lab 01

Ej: ssh root@name.sco.alma.cl

Login into your docker host machine

Lab 01

- docker version

- docker ps

- docker images

- etc...

Verify docker is installed

Lab 01

  1. cd /etc/offline/base_image
  2. docker build -t ioffline .

Create a new docker image

Lab 01

docker run -d --name f2f-XX -v /etc/offline/:/etc/offline/ -p 8080:8080 ioffline:latest

Run a new "offline" container

Lab 01

(verify that the container is running)

docker exec -i -t f2f-XX /bin/bash

Enter into the running container

Lab 01

supervisorctl stop tomcat

 

Stop Tomcat (using supervisord)

Lab 01

  1. cp /etc/offline/wars/protrack.war /usr/share/tomcat/webapps/
  2. supervisorctl start tomcat

Install Protrack war

Lab 01

https://f2f-XX.asa-test.alma.cl/protrack

View the application running

Lab 01

Lab 01

Docker Host

f2f-workshop.sco.alma.cl

Ubuntu 14.04

Docker

Container

(Fedora)

XX.sco.alma.cl

10.200.67.XX

Ports: 22, 8080

Docker Host

 

Docker

Container

(RHEL 6.6)

Docker

Container

(Fedora)

XX.sco.alma.cl

10.200.67.XX

Ports: 22, 8080

Docker Host

 

Docker

Container

(RHEL 6.6)

Docker

Container

(Fedora)

XX.sco.alma.cl

10.200.67.XX

Ports: 22, 8080

Docker Host

 

Docker

Container

(RHEL 6.6)

Docker

Container

(Fedora)

XX.sco.alma.cl

10.200.67.XX

Ports: 22, 8080

Docker Host

 

Docker

Container

(RHEL 6.6)

Lab 02

http://www.docker.com/toolbox

 

Mac:            https://download.asa-test.alma.cl/DockerToolbox-1.8.2c.pkg

Windows:   https://download.asa-test.alma.cl/DockerToolbox-1.8.2c.exe

Installing Docker on Mac (and Windows)

https://docs.docker.com/installation/

Installing Docker Linux

Lab 02

https://svn.alma.cl/p2/trunk/ADC/SW/OFFLINE/F2f-workshop/config/

Example config files

https://svn.alma.cl/p2/trunk/ADC/SW/OFFLINE/F2f-workshop/wars/

 

 

svn co https://svn.alma.cl/p2/trunk/ADC/SW/OFFLINE/F2f-workshop/

Example war files

Lab 02

docker pull docker-registry.asa-test.alma.cl/ioffline

Download ioffline image from registry

Lab 02

docker run -d --name app -v /path-to/F2f-workshop/:/etc/offline/  -p 8080:8080 ioffline:latest

Run container

Lab 02

docker exec -i -t app /bin/bash

Access to container and install war

Lab 02

Made with Slides.com