Provision a Ready to GO Azure Server

Hatem Ben T yeb

Devops Engineer @ Astrolab-agency Sousse

linkedin.com/in/hatembentayeb/

facebook.com/htayeb2

Opensource Guy, Automation Enthusiast, Archlinux SuperFan

Architecture

Pipeline

Run Pipeline
Auth to azure
Spin VM
Check VM state
Get VM IP
Configure VM

Tools

Docker : Custom Image that contains all necessary tools.
Gitlab: It's our CI/CD server.
Ansible: Spin and configure ou Server.
Bash : Writing shell scripts.
Az : Azure cli for authentication and VMs info.

Deep Dive

Push Vars.conf

Vars.conf is a configuration file that holds all your Server configuration like name, ssh-key, subnet, resource-group ...etc

RESSOURCE_GROUP_NAME="hello-server"
LOCATION="westus2"
VIRTUAL_NETWORK_NAME="hello-server-vnetwork"
VN_IP_PREFIX="10.0.0.0/16"
SUBNET_NAME="hello-server-subnet"
SN_IP_PREFIX="10.0.1.0/24"
PUBLIC_IP_NAME="hello-server-ip"
SECURITY_GROUP_NAME="hello-server-sg"
VIRTUAL_NETWORK_INTERFACE_NAME="hello-server-nic"
VIRTUAL_MACHINE_NAME="hello-server-vm"
VIRTUAL_MACHINE_SIZE="Standard_F2s_v2"
ADMIN_USERNAME="hello-server"
SSH_PUBLIC_KEY_PATH="/home/hello-server/.ssh/authorized_keys"
IMAGE_OFFER="UbuntuServer"
IMAGE_PUBLISHER="Canonical"
IMAGE_SKU='18.04-LTS'
IMAGE_VERSION="latest"

Run !

The pipeline cannot be launched automatically on push .. but only with specific parameters. either on gitlab or by a remote request with CURL


curl \
   --request POST \
  --form token=TOKEN \                                                                            
  --form ref=<branch> \
  --form "variables[VM_PROVISION]=true" \
  --form "variables[VM_CONFIG]=true"  \
  
  https://gitlab.com/api/v4/projects/<project-repo-id>/trigger/pipeline | jq

Looking Inside

Dev

Ops

Pipeline

The Gitlab pipeline is splitted into two main stages "spinup_vm" and "configure_m".

variables :
  VM_PROVISION: "false"
  VM_CONFIG: "false"
  VALIDATE_PLAYBOOKS: "false"

stages: 
  - linting_playbooks
  - spinup_vm
  - configure_vm

# my customized image for the pipeline : hatembt/azure-ci (available in public on dockerhub)

linting_playbooks:
  stage: linting_playbooks
  image: hatembt/azure_ci:1.0
  script:
    - ansible-lint -x command-shell,formatting,repeatability spin_vm/*.yml

  tags:
    - gitlab-org-docker
  only:
    refs:
      - master
    variables:
      - $VALIDATE_PLAYBOOKS == "true"

spinup_vm:
  stage: spinup_vm
  image: hatembt/azure_ci:1.0
  variables:
    DESIRED_STATE: "running"
  before_script:
    - ./azure_login.sh
  script:
    - set -o allexport
    - source vars.conf 
    - set +o allexport
    - ansible-playbook spin_vm/provision_vm.yml
  after_script:
    - set -o allexport
    - source vars.conf 
    - set +o allexport
    - ./check_vm_state.sh

  tags:
    - gitlab-org-docker
  only:
    refs:
      - master
    variables:
      - $VM_PROVISION == "true"

configure_vm:
  stage: configure_vm
  image: hatembt/azure_ci:1.0
  before_script:
    - pip install netaddr paramiko
    - ./azure_login.sh
    - set -o allexport
    - source vars.conf 
    - set +o allexport
  script:
    #- cat ${SSH_CFG} > "$CI_PROJECT_DIR/ssh.cfg"  
    - cd $CI_PROJECT_DIR && ansible-playbook  config_vm/inventory_setup.yml  
    - mkdir -p "$CI_PROJECT_DIR/.ssh"                            
    - cat ${PRIVATE_KEY} > "$CI_PROJECT_DIR/.ssh/id_rsa"      
    - chmod -R 600 "$CI_PROJECT_DIR/.ssh/id_rsa" 
    - export ANSIBLE_HOST_KEY_CHECKING=False   
    - cd $CI_PROJECT_DIR && ansible-playbook -i config_vm/inventory_bastion   config_vm/configure_bastion.yml --private-key="$CI_PROJECT_DIR"/.ssh/id_rsa
  after_script:
    - rm -r "$CI_PROJECT_DIR/.ssh" || true                              
  tags :
    - gitlab-org-docker
  only:
    refs:
      - master
    variables:
      - $VM_CONFIG == "true"

The pipeline stages are based on the image "hatembt/azure_ci:1.0"

Playbooks

Playbooks are written as tasks and then we include them on the main playbook to ensure flexibility

- name: configure VM
  hosts: azure 
  gather_facts: no
  become: yes
  tasks:

  # - import_tasks: playbooks/install_git.yml
  # - import_tasks: playbooks/install_nodejs-12-lts.yml
  # - import_tasks: playbooks/install_pm2.yml
  - import_tasks: playbooks/install_nginx.yml
  - import_tasks: playbooks/configure_nginx.yml
  - import_tasks: playbooks/install_certbot.yml
  - import_tasks: playbooks/install_docker.yml

Templates

Templates are based on {{ jinja2 }} syntax.

- name: generate inventory
  hosts: localhost
  connection : local
  vars:
    bastion_rg: "{{ lookup('env','RESSOURCE_GROUP_NAME') }}"
    bastion_name:  "{{ lookup('env','VIRTUAL_MACHINE_NAME') }}"
    user: "{{ lookup('env','ADMIN_USERNAME') }}"

  tasks:
    - name : get bastion IP
      script : ./get_vm_ip.sh "{{ bastion_rg }}" "{{ bastion_name }}"
      register: ip_adderr
    - name: store ip address
      set_fact:
        ip_address={{ ip_adderr.stdout }}

    - name: templating inventory
      template:
         src: inventory.j2
         dest: inventory_bastion

This playbook will generate an inventory file to be used by the main playbook to configure the Server

Screenshots

After Success of the two stages

we can check the server by

Accessing IP address via port 80 and we should get nginx welcome page
SSH with user, IP and private key

WrapUp

Easy to use
Flexible
Extendible
Easy to maintain
Fast
Reliable