Gitlab-ci & IaC sharing

Outline

  • About me
  • Gitlab-ci
  • IaC
  • Case Experience

About Me

Gitlab-ci

GitLab CI/CD workflow

Adventage

  • Easy to setup
  • No third party plugin is needed
  • Fewer cost to manage

How It Work

How To Trigger

  • edit .gitlab-ci.yml
  • define stages
  • update branch

Runner Execution Flow

.gitlab-ci.yml layout

stages:
  - build
  - test

build-code-job:
  stage: build
  script:
    - echo "Check the ruby version, then build some Ruby project files:"
    - ruby -v
    - rake

test-code-job1:
  stage: test
  script:
    - echo "If the files are built successfully, test some files with one command:"
    - rake test1

test-code-job2:
  stage: test
  script:
    - echo "If the files are built successfully, test other files with a different command:"
    - rake test2

Job Running

Pipeline Status

More Options

job:
  # use regexp
  image: golang:alpine
  script:
   - |
   	printenv
  only:
    - /^issue-.*$/
  # use special keyword
  except:
    - branches
  rules:
    - if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
      when: manual
      allow_failure: true
    - if: '$CI_PIPELINE_SOURCE == "schedule"'

What if I want to test a single stage?

git cibr ${branch_name}
[alias]
  cibr = !export cur_br=$1 && echo $1 && git branch ${cur_br} && 
  git push origin ${cur_br} && sleep 30 && git push origin --delete ${cur_br} && 
  git branch -d ${cur_br} && echo "done"

IaC

Infrastructure as Code

Advantage

  • gui的更新可能造成操作失誤  
  • 操作快速
  • 方便管理

IaC tools comparison

tools Pulumi Terraform Cloudformation
language JS/Go/python... HCL Json/Yaml
Cloud All All AWS
Source Open Open Close
Rollback ? No Yes

Cloudformation Layout


AWSTemplateFormatVersion: "2010-09-09"
Description: A sample template
Resources:
  MyEC2Instance: #An inline comment
    Type: "AWS::EC2::Instance"
    Properties: 
      ImageId: "ami-0ff8a91507f77f867" #Another comment -- This is a Linux AMI
      InstanceType: t2.micro
      KeyName: testkey
      BlockDeviceMappings:
        -
          DeviceName: /dev/sdm
          Ebs:
            VolumeType: io1
            Iops: 200
            DeleteOnTermination: false
            VolumeSize: 20

Terraform Layout

terraform {
  backend "s3" {
    bucket = "mybucket"
    key    = "path/to/my/key"
    region = "us-east-1"
  }
}

provider "aws" {
  profile    = "default"
  region     = "us-east-1"
}

resource "aws_instance" "example" {
  ami           = "ami-2757f631"
  instance_type = "t2.micro"
}

Case Experience

Migration via Cloud

Scenario

Drawbacks

  • manual can make mistakes
  • too many steps
  • need jumper

Solution

  1. Lambda instead of jumper
  2. EC2 and SSM document

Solution1

Lambda instead of jumper

以Gitlab-ci搭配cloudformation示範

image: amazon/aws-sam-cli-build-image-python3.8

variables:
  HANDLER: handler
  
stages:
  - deploy

update_lambda:
  stage: deploy
  script:
    - |
      sam build --template-file template.yaml
      sam package --region ap-northeast-1 --resolve-s3
      sam deploy \
          --capabilities CAPABILITY_IAM CAPABILITY_NAMED_IAM \
          --region ap-northeast-1 \
          --stack-name $BRANCH-migration \
          --resolve-s3 \
          --parameter-overrides Handler=${HANDLER}
  only:
    - test
    - qa
    - stag
    - prep
    - prod
    - cust

Gitlab-ci.yml for lambda example

Adventage

  • Lambda的高併發快速且節省成本
  • 不須jumper
  • 容易管理

Disadvantage

  • 無法generic,每個lambda都只能做migration一件事
  • lambda有15分鐘timeout限制

Solution2

EC2 and SSM document

以Gitlab-ci搭配terraform示範

Adventage

  • 一個instance可同時負責一個env所有類型的部屬
  • 沒有jumper
  • 一份document可同時給不同環境使用

Disadventage

  • instance成本較lambda高

template

reference

cicd

By nathanlin

cicd

  • 581