Provision a Ready to GO Azure Server

Hatem Ben T  yeb

Devops Engineer @ Astrolab-agency Sousse

linkedin.com/in/hatembentayeb/

facebook.com/htayeb2

Opensource Guy, Automation Enthusiast, Archlinux SuperFan

Architecture

Pipeline

  • Run Pipeline
  • Auth to azure
  • Spin VM
  • Check VM state
  • Get VM IP
  • Configure VM

Tools

  • Docker : Custom Image that contains all necessary tools.
  • Gitlab: It's our CI/CD server. 
  • Ansible: Spin and configure ou Server.
  • Bash : Writing shell scripts.
  • Az : Azure cli for authentication and VMs info.

Deep Dive

Push Vars.conf

Vars.conf is a configuration file that holds all your Server configuration like name, ssh-key, subnet, resource-group ...etc

RESSOURCE_GROUP_NAME="hello-server"
LOCATION="westus2"
VIRTUAL_NETWORK_NAME="hello-server-vnetwork"
VN_IP_PREFIX="10.0.0.0/16"
SUBNET_NAME="hello-server-subnet"
SN_IP_PREFIX="10.0.1.0/24"
PUBLIC_IP_NAME="hello-server-ip"
SECURITY_GROUP_NAME="hello-server-sg"
VIRTUAL_NETWORK_INTERFACE_NAME="hello-server-nic"
VIRTUAL_MACHINE_NAME="hello-server-vm"
VIRTUAL_MACHINE_SIZE="Standard_F2s_v2"
ADMIN_USERNAME="hello-server"
SSH_PUBLIC_KEY_PATH="/home/hello-server/.ssh/authorized_keys"
IMAGE_OFFER="UbuntuServer"
IMAGE_PUBLISHER="Canonical"
IMAGE_SKU='18.04-LTS'
IMAGE_VERSION="latest"

Run !

The pipeline cannot be launched automatically on push .. but only with specific parameters. either on gitlab or by a remote request with CURL


curl \
   --request POST \
  --form token=TOKEN \                                                                            
  --form ref=<branch> \
  --form "variables[VM_PROVISION]=true" \
  --form "variables[VM_CONFIG]=true"  \
  
  https://gitlab.com/api/v4/projects/<project-repo-id>/trigger/pipeline | jq

Looking Inside

Dev

Ops

Pipeline 

The Gitlab pipeline is splitted into two main stages "spinup_vm" and "configure_m".

variables :
  VM_PROVISION: "false"
  VM_CONFIG: "false"
  VALIDATE_PLAYBOOKS: "false"

stages: 
  - linting_playbooks
  - spinup_vm
  - configure_vm

# my customized image for the pipeline : hatembt/azure-ci (available in public on dockerhub)

linting_playbooks:
  stage: linting_playbooks
  image: hatembt/azure_ci:1.0
  script:
    - ansible-lint -x command-shell,formatting,repeatability spin_vm/*.yml

  tags:
    - gitlab-org-docker
  only:
    refs:
      - master
    variables:
      - $VALIDATE_PLAYBOOKS == "true"

spinup_vm:
  stage: spinup_vm
  image: hatembt/azure_ci:1.0
  variables:
    DESIRED_STATE: "running"
  before_script:
    - ./azure_login.sh
  script:
    - set -o allexport
    - source vars.conf 
    - set +o allexport
    - ansible-playbook spin_vm/provision_vm.yml
  after_script:
    - set -o allexport
    - source vars.conf 
    - set +o allexport
    - ./check_vm_state.sh

  tags:
    - gitlab-org-docker
  only:
    refs:
      - master
    variables:
      - $VM_PROVISION == "true"

configure_vm:
  stage: configure_vm
  image: hatembt/azure_ci:1.0
  before_script:
    - pip install netaddr paramiko
    - ./azure_login.sh
    - set -o allexport
    - source vars.conf 
    - set +o allexport
  script:
    #- cat ${SSH_CFG} > "$CI_PROJECT_DIR/ssh.cfg"  
    - cd $CI_PROJECT_DIR && ansible-playbook  config_vm/inventory_setup.yml  
    - mkdir -p "$CI_PROJECT_DIR/.ssh"                            
    - cat ${PRIVATE_KEY} > "$CI_PROJECT_DIR/.ssh/id_rsa"      
    - chmod -R 600 "$CI_PROJECT_DIR/.ssh/id_rsa" 
    - export ANSIBLE_HOST_KEY_CHECKING=False   
    - cd $CI_PROJECT_DIR && ansible-playbook -i config_vm/inventory_bastion   config_vm/configure_bastion.yml --private-key="$CI_PROJECT_DIR"/.ssh/id_rsa
  after_script:
    - rm -r "$CI_PROJECT_DIR/.ssh" || true                              
  tags :
    - gitlab-org-docker
  only:
    refs:
      - master
    variables:
      - $VM_CONFIG == "true"

The pipeline stages are based on the image "hatembt/azure_ci:1.0"

Playbooks 

Playbooks are written as tasks and then we include them on the main playbook to ensure flexibility 

- name: configure VM
  hosts: azure 
  gather_facts: no
  become: yes
  tasks:

  # - import_tasks: playbooks/install_git.yml
  # - import_tasks: playbooks/install_nodejs-12-lts.yml
  # - import_tasks: playbooks/install_pm2.yml
  - import_tasks: playbooks/install_nginx.yml
  - import_tasks: playbooks/configure_nginx.yml
  - import_tasks: playbooks/install_certbot.yml
  - import_tasks: playbooks/install_docker.yml

Templates

Templates are based on {{ jinja2 }} syntax.

- name: generate inventory
  hosts: localhost
  connection : local
  vars:
    bastion_rg: "{{ lookup('env','RESSOURCE_GROUP_NAME') }}"
    bastion_name:  "{{ lookup('env','VIRTUAL_MACHINE_NAME') }}"
    user: "{{ lookup('env','ADMIN_USERNAME') }}"

  tasks:
    - name : get bastion IP
      script : ./get_vm_ip.sh "{{ bastion_rg }}" "{{ bastion_name }}"
      register: ip_adderr
    - name: store ip address
      set_fact:
        ip_address={{ ip_adderr.stdout }}

    - name: templating inventory
      template:
         src: inventory.j2
         dest: inventory_bastion 

This playbook will generate an inventory file to be used by the main playbook to configure the Server

Screenshots

After Success of the two stages  

we can check the server by 

  • Accessing IP address via port 80 and we should get nginx welcome page 
  • SSH with user, IP and private key 

WrapUp

  • Easy to use
  • Flexible 
  • Extendible
  • Easy to maintain
  • Fast 
  • Reliable

Provision a ready to Go Azure server

By hatem ben tayeb

Provision a ready to Go Azure server

The laziest way to provison a server :)

  • 592