Abdullah Fathi
Pautan Muat Turun
Recap: Agile
Pasukan Pembangun
Pasukan Operasi
Agile + DevOps
"DevOps"
Proses yang Sama: Deliver aplikasi kepada pengguna akhir
Sentiasa menjadi matlamat utama
Deliver aplikasi kepada pengguna
Application Release Process
Release to Market
Development
Operations
Same Process of: Deliver application to the end user
This is always the main goal!
Deliver application to end users
Typical Software Release Process
Idea
Requirements
Code it
Test it
Build & Package
Deploy it
Configure the server
Typical Software Release Process
Idea
Requirements
Code it
Test it
Build & Package
Deploy it
Operate & Monitor
Typical Software Release Process
Initial Launch
Improvements
Idea
Requirements
Code it
Test it
Build & Package
Deploy it
Operate & Monitor
Make it accessible to user immediately
Software versioning
Code it
Test it
Build & Package
Version 1
Code it
Test it
Build & Package
Version 2
Code it
Test it
Build & Package
Version 3
Code it
Test it
Build & Package
Version 4
Replaced Framework
New Feature
1.0.0
1.1.0
1.1.1
1.1.2
1 . 5 . 2
Major
Minor
Patch
Bug fix
Bug fix
Delivery Process
Implement
Idea
Test
Build
Deploy
Observe
Quickly Delivering with High Quality Code
Test it
Build & Package
Deploy it
Push Code
Changes
Artifact
Repository
CI
CD
CD of CI/CD
Deploy to
PROD
Continuous Deployment
Deploy to
PROD
Deploy to
DEV
Deploy to
STAGING
Unit Tests,
functional tests
Code analysis tests, lint tests
Load tests,
Regression tests
Continuous Delivery
Continuous Delivery
Deploy to
PROD
Deploy to
DEV
Deploy to
STAGING
Unit Tests,
functional tests
Code analysis tests, lint tests
Load tests,
Regression tests
CI of CI/CD
remote git
repository
Test it
Build & Package
Push Code
Changes
Continuous Integration
Why continuously integrating the code changes is important?
Rangka Kerja DevOps Sektor Awam - Muka Surat: 75
Infrastruktur Pipeline DevOps
Tools DevOps
Git Introduction
Why Git?
Git Advantages vs Disadvantages
# Check Git Version
git --version
# Create a new empty git repository
git init
# Create git repository
git init <project-name>
# Configure initial branch name to
# use in all of your new repositories
git config --global init.defaultBranch main
# Rename the branch of current git repository to main
git branch -m main
# Verify folder is git repository
# Tells you the current status of git repository
git status
# Set author username
git config --global user.name "USERNAME"
# Verify configured user name
git config --global user.name
# Set email address
git config --global user.email "test@fotia.com.my"
# Verify configured email address
git config --global user.email
Add git username/email to identify the author
Review History of Repository
# Show a history of changes
git log
git log --all --oneline
git log --patch
# State changes for commit
git add .
# or add multiple selected files
git add 'file1' 'file2' 'file3'
# Commit staged changes
git commit -m "message-here"
Commit
working directory
staging area
repository
git add
git commit
# Create branch
git branch <branch-name>
# or create branch and checkout to that branch
$ git checkout -b <branch-name>
# View Branches
git branch
# Switch branch
git checkout <branch-name-here>
Branch
Rebase, Resolve Conflict
# Rebase to sync with the master/main branch
git rebase master
# Resolve conflict when merge request
(on branch feature)
git checkout main
git pull origin main
git checkout <branch_feature>
git rebase main
(solve conlict)
git add .
git commit -m "solve conflict"
git rebase --continue
git push origin feature --force
# Resolve Conflict when merge request
git checkout main
git pull origin main
git checkout <branch_feature>
git merge main (conflict)
<solve conflict>
git add .
git commit -m "message"
git push origin <branch_feature>
# Abort merge conflict
git merge --abort
# Push file to stash
git stash push
# List stash files
git stash list
# Reapply change to working directory
# Does not remove this change from the stash
git stash apply
# Pop change of the stash
# Automatically remove from stash list
git stash pop
# Clear stash
git stash clear
Stash
Git
Version Control System
Gitlab
Source Code Management System
Git VS Gitlab
Gitlab Overview
Why Gitlab?
User Profile -> preferences -> SSH Keys
ssh-keygen -t ed25519 -C "Gitlab Key Pair"
# test authenticate to gitlab account
ssh -T git@gitlab.com
SSH is a way to authenticate without exposing your username and password
SSH Keys
User Profile -> preferences -> Access Tokens
git clone https://<git-project-name-here>:<access-token-here>@<git-project-url-here>
Authenticate with Personal Access Token in place of your password
Access Token
Gitlab: Group
Gitlab: Project
Gitlab: Member
Gitlab: Merge Request
Gitlab: Issue Tracking
Home Page
Profile
Menu -> Groups -> Create Group
Manage one or more related projects at the same time
Create Group
Groups
Menu -> Projects -> Create New Project
Files and code are saved in projects, and most features are in the scope of projects
Create New Blank Project
Initialize repo with readme file if there is no existing code base
Projects
Side Menu-> Issues -> Milestones
A way to track issue and merge requests created to achieve a broader goal in a certain period of time
Can be used as:
Milestones
Use issues to collaborate on ideas, solve problems, and plan work.
Purpose of Issue:
Side Menu-> Issues
Issues
Provide efficient email support to customers
Type of email for service desk:
Side Menu-> Issues -> Service Desk
Custom Email:
Settings -> General -> Service Desk
Service Desk
Project Documentation
Gitlab wiki content format:
Side Menu-> Wiki
Wiki
Store and share bits of code and text with other users
Two types of snippets:
Side Menu-> Snippets
Snippets
One platform for all your DevOps capabilities
JENKINS: Currently the most well-known and widely used tool for building CI/CD pipelines
Branching Strategy
Gitlab Primary Branching Strategy
2 Variation:
Gitlab Flow
Gitlab Flow (Environment Branches)
Gitlab Flow (Release Branches)
The three primary approaches for the continuous method :-
2. Continuous Delivery
Not only is your application built and tested each time a code change is pushed to the codebase, the application is also deployed continuously. However, with continuous delivery, you trigger the deployments manually.
3. Continuous Deployment
Similar to Continuous Delivery. The difference is that instead of deploying your application manually, you set it to be deployed automatically. Human intervention is not required
Configuration File
What is Gitlab Runner?
How do we build connection between gitlab runner server and gitlab instance server?
By register installed runner to gitlab server
Gitlab Runner
A runner can be:
Gitlab Runner
Shared VS Specific Runner
Tagged VS Untagged
Protected VS Unprotected
Runner Executor
CI/CD Pipeline
Jobs
Basic building block of pipeline
job
job
job
job
Pipeline
.gitlab-ci.yml
Jobs
before_script
is used to define a command that should be run before each job, including deploy jobs, but after the restoration of any artifacts
after_script
is used to define the command that will be run after each job, including failed ones.
Run before and after the script defined in each job
before_script and after_script
●image
●services
●script
●before_script & after_script
●variables
●Environment
●cache
●artifacts
●rules
●tags
●when
Stages
Group jobs
User can define custom stages & any number of jobs per stage
Stages seperate jobs into logical sections while Jobs perform the actual tasks
Stages
Needs
Relationship between jobs
Allow Failure
Keyword to continue the pipeline execution and report a success even if the job fails
"allow_failure:true"
Script
Inline shell commands and executing shell script
Script
Jobs and Scripts
Only/Except
The rules syntax is an improved, more powerful solution for defining when jobs should run or not. Consider using rules instead of only/except to get the most out of your pipelines.
Only/Except
Basic Parameter
Predefined Variables
Predefined CI/CD variables are available in every Gitlab CI/CD pipeline
Workflow Rules
"workflow"
A global keyword, which configures the whole pipelines behavior
Workflow Rules
"workflow:rules"
Rules
Common Rules Use Cases
Rules
When is a Job created in Pipeline?
Rules Syntax
when Options
on_success
(default): Run the job only when no jobs in earlier stages fail or have allow_failure: true
.on_failure
: Run the job only when at least one job in an earlier stage fails. A job in an earlier stage with allow_failure: true
is always considered successful.never
: Don’t run the job regardless of the status of jobs in earlier stages. Can only be used in a rules
section or workflow: rules
.always
: Run the job regardless of the status of jobs in earlier stages. Can also be used in workflow:rules
.manual
: Run the job only when triggered manually.delayed
: Delay the execution of a job for a specified duration.Rules
When is a Job NOT created in Pipeline?
Manual Execution
Rules Examples 1
Rules Examples 2
Rules Examples 3
Rules Examples 4
Variables
Variables Processing Order
Templates are 0% magical, they are GitLab CI/CD YAML files
Templates are always executed into a CI/CD pipeline through an include statement in the project’s .gitlab-ci.yml
Template jobs are created in your CI/CD pipeline based on their defined stage (and any applicable rules)
Template
Template
To take advantage of SAST capabilities from GitLab engineering, we will use include:template
Template
Share Pipeline Definitions
Share Pipeline Definitions
Share Pipeline Definitions
Extending Jobs
Example Using extends
Artifacts
Environment
Preparation
Cache and Artifacts Download
Running
Scripts
Cache and Artifacts Upload
Cleanup
Gitlab job execution phases
Artifact Download
Example Build Artifact
Artifact VS Cache
Image
Environments
The environment keyword defines where the app is deployed and is defined by 3 parts.
When triggers jobs & stages manually (e.g. deploy to production)
Environments
There are multiple type of testing that we can execute on stage Test
●
●
Gitlab SAST use analyzer to check source code for any known vulnerabilities
include:
- template: Jobs/SAST.gitlab-ci.yml
#Static Application Security Test (SAST)
sast:
stage: test
artifacts:
paths: [gl-sast-report.json]
reports:
sast: gl-sast-report.json
Built on top of Elastic Stack to monitor application performance and services
Monolithic Architecture
Challenges of Monolithic Architecture
Microservices
Communication between Microservices
Challenges of Microservices Architecture
Monorepo
MS
MS
MS
MS
MS
MS
Microservices
Monorepo
1 Git repository that contains many projects
Monorepo
User
Thread
Post
Monorepo
Monorepo Challenges
Polyrepo
User
Polyrepo
Thread
Post
Polyrepo Challenges
User
Thread
Post
Monorepo
User
Polyrepo
Thread
Post
For smaller microservice application
Polyrepo VS Monorepo
Download Presentation Slide
Download Presentation Slide