Abdullah Fathi
Pautan Muat Turun
Recap: Agile
Penyelesaiannya adalah DevOps
Pasukan Pembangun
Pasukan Operasi
Agile + DevOps
Currently the most heard buzz word in IT industry
- Relatively new concept
- Has been gaining popularity
"DevOps"
Proses yang Sama: Deliver aplikasi kepada pengguna akhir
Sentiasa menjadi matlamat utama
Deliver aplikasi kepada pengguna
Application Release Process
Release to Market
Development
Operations
Same Process of: Deliver application to the end user
This is always the main goal!
Deliver application to end users
Typical Software Release Process
Idea
Requirements
Code it
Test it
Build & Package
Deploy it
- Installing tools
- Deploy application
- Configure firewall
Configure the server
Typical Software Release Process
Idea
Requirements
Code it
Test it
Build & Package
Deploy it
- Any problems with the application?
- Are users experiencing any issues?
- Can the application handle high user load?
Operate & Monitor
Typical Software Release Process
- Add new features!
- Fix bugs
- Optimize performance
Initial Launch
Improvements
Idea
Requirements
Code it
Test it
Build & Package
Deploy it
Operate & Monitor
Make it accessible to user immediately
Software versioning
Code it
Test it
Build & Package
Version 1
Code it
Test it
Build & Package
Version 2
Code it
Test it
Build & Package
Version 3
Code it
Test it
Build & Package
Version 4
Replaced Framework
New Feature
1.0.0
1.1.0
1.1.1
1.1.2
1 . 5 . 2
Major
Minor
Patch
Bug fix
Bug fix
Delivery Process
Implement
Idea
Test
Build
Deploy
Observe
- Making the process fast
- Minimal Bugs
Quickly Delivering with High Quality Code
CI/CD is in the heart of DevOps
Test it
Build & Package
Deploy it
Push Code
Changes
Artifact
Repository
CI
CD
CD of CI/CD
Deploy to
PROD
- We don't deploy directly to PRODUCTION environment
- We can't be sure that the changes won't break anything
- We deploy the application in stages
- To test the application extensively on multiple levels
Continuous Deployment
Deploy to
PROD
Deploy to
DEV
Deploy to
STAGING
Unit Tests,
functional tests
Code analysis tests, lint tests
Load tests,
Regression tests
- Every code change is built, tested and deployed to the different environments
- Deployment to production happens automatically without explicit manual approval
Continuous Delivery
- Automated test not 100% reliable
- Not enough time or too difficult to test each and every aspect of the application
- Not confident enough to release automatically to production
Continuous Delivery
Deploy to
PROD
Deploy to
DEV
Deploy to
STAGING
Unit Tests,
functional tests
Code analysis tests, lint tests
Load tests,
Regression tests
- Pipeline is automated only until the non-production environment
- Update to production happens only with a manual approval
CI of CI/CD
remote git
repository
Test it
Build & Package
Push Code
Changes
Continuous Integration
- Developer commits code changes to remote git repository
- Triggers the CI/CD Pipeline
Why continuously integrating the code changes is important?
- Each developer works with their own local copy of app code
- Changes need to be integrated ("merged") into central shared code
- Handle conflict, if some code updated
- Run automated tests: The bigger the code changes, the more likely that something in the application breaks
Rangka Kerja DevOps Sektor Awam - Muka Surat: 75
Infrastruktur Pipeline DevOps
Tools DevOps
Gitlab The DevOps Platform
-
Git: Version Control System for:
- Tracking code changes
- Tracking who made changes
- Coding collaboration
- Gitlab, Github, Bitbucket: Services that provide remote access to git repositories
Git Introduction
- Projects generally have multiple developers working in parallel, Git is needed to ensure there are no code conflicts between the developers
- The requirements in such projects change often. So a version control system allows developers to revert and go back to an older version of the code.
- Sometimes several projects which are being run in parallel involve the same codebase. In such a case, the concept of branching in Git is very important.
Why Git?
Advantages
- Good distributed model
- Branching capabilities and merging are easy
- Free and open-source
- Good and faster network performance and superior disk utilization
Disadvantages
- Poor GUI and usability
- GIT needs multiple branches to support parallel developments used by the developers
Git Advantages vs Disadvantages
Git Installation
Git Commands
# Check Git Version
git --version
# Create a new empty git repository
git init
# Create git repository
git init <project-name>
# Configure initial branch name to
# use in all of your new repositories
git config --global init.defaultBranch main
# Rename the branch of current git repository to main
git branch -m main
# Verify folder is git repository
# Tells you the current status of git repository
git status
Git Commands
# Set author username
git config --global user.name "USERNAME"
# Verify configured user name
git config --global user.name
# Set email address
git config --global user.email "test@fotia.com.my"
# Verify configured email address
git config --global user.email
Add git username/email to identify the author
Review History of Repository
# Show a history of changes
git log
git log --all --oneline
git log --patch
Git Commands
# State changes for commit
git add .
# or add multiple selected files
git add 'file1' 'file2' 'file3'
# Commit staged changes
git commit -m "message-here"
Commit
working directory
staging area
repository
git add
git commit
Git Commands
# Create branch
git branch <branch-name>
# or create branch and checkout to that branch
$ git checkout -b <branch-name>
# View Branches
git branch
# Switch branch
git checkout <branch-name-here>
Branch
Git Commands
Rebase, Resolve Conflict
# Rebase to sync with the master/main branch
git rebase master
# Resolve conflict when merge request
(on branch feature)
git checkout main
git pull origin main
git checkout <branch_feature>
git rebase main
(solve conlict)
git add .
git commit -m "solve conflict"
git rebase --continue
git push origin feature --force
# Resolve Conflict when merge request
git checkout main
git pull origin main
git checkout <branch_feature>
git merge main (conflict)
<solve conflict>
git add .
git commit -m "message"
git push origin <branch_feature>
# Abort merge conflict
git merge --abort
Git Commands
# Push file to stash
git stash push
# List stash files
git stash list
# Reapply change to working directory
# Does not remove this change from the stash
git stash apply
# Pop change of the stash
# Automatically remove from stash list
git stash pop
# Clear stash
git stash clear
Stash
Git
Version Control System
Gitlab
Source Code Management System
Git VS Gitlab
- Open Source software development platform
- Source Code Management System
- CI/CD
- Complete DevOps platform
Gitlab Overview
- Host your application's code
- Enables collaboration among a team of developers with Gitlab Flow
- Built-in CI/CD functionality
- Can integrate with other tools
- Granular access controls
- File locking, preventing conflicts
- Many more other great features..
Why Gitlab?
User Profile -> preferences -> SSH Keys
ssh-keygen -t ed25519 -C "Gitlab Key Pair"
# test authenticate to gitlab account
ssh -T git@gitlab.com
SSH is a way to authenticate without exposing your username and password
SSH Keys
User Profile -> preferences -> Access Tokens
git clone https://<git-project-name-here>:<access-token-here>@<git-project-url-here>
Authenticate with Personal Access Token in place of your password
Access Token
- Manage settings across multiple projects
- Logically categorize multiple users or projects
- Provide a cross-project view
Gitlab: Group
- A container for a Git repository
- Built in CI/CD
- Issue Tracking
- Collaboration Tools
Gitlab: Project
- Users or Groups that have access to a project
- Members are assigned to roles
- Member roles include permissions to perform actions on projects or groups
Gitlab: Member
- A request to merge one branch into another
- Merge Requests provide a space to have a conversation with the team about the changes on a branch
- It is the central place through which changes are reviewed and verified
- Merge Request Role:
- Assignee: A person who is directly responsible for creating or merging a merge request
- Reviewer: A team member who may only be involved in one aspect of the merge request, such as a peer review
Gitlab: Merge Request
- An issue is a way to track work related to a Gitlab project
- Issues can be used to report bugs, track tasks, request new features, ask a question and more
- Your development workflow should begin with an issue
Gitlab: Issue Tracking
Home Page
Profile
Menu -> Groups -> Create Group
Manage one or more related projects at the same time
Create Group
Groups
Menu -> Projects -> Create New Project
Files and code are saved in projects, and most features are in the scope of projects
Create New Blank Project
Initialize repo with readme file if there is no existing code base
Projects
Side Menu-> Issues -> Milestones
A way to track issue and merge requests created to achieve a broader goal in a certain period of time
Can be used as:
- Agile Sprint
- Releases
Milestones
Use issues to collaborate on ideas, solve problems, and plan work.
Purpose of Issue:
- Discuss the implementation of an idea.
- Track tasks and work status.
- Accept feature proposals, questions, support requests, or bug reports.
- Elaborate on code implementations.
Side Menu-> Issues
Issues
Provide efficient email support to customers
Type of email for service desk:
- Bug Reports
- Feature Request
- General Feedback
Side Menu-> Issues -> Service Desk
Custom Email:
Settings -> General -> Service Desk
Service Desk
Project Documentation
Gitlab wiki content format:
- Markdown
- RDoc
- AsciiDoc
- Org
Side Menu-> Wiki
Wiki
Store and share bits of code and text with other users
Two types of snippets:
- Personal Snippets
- Project Snippets
(Depends on Project Visibility)
Side Menu-> Snippets
Snippets
CI/CD Platform
(Gitlab)
- Integrate various tools to do the testing, building, etc.
- Platform to build CI/CD Pipeline
- Covering the complete software development and release cycle
One platform for all your DevOps capabilities
Gitlab CI/CD
VS
Jenkins
JENKINS: Currently the most well-known and widely used tool for building CI/CD pipelines
- Still an industry leader
- Powerful, Open source, big community
- Flexible integrations through 1000s of community plugin
- Old tool, which was not designed for the new container age
- Need to install, configure or maintain additional plugins
- Gitlab is a DevOps platform
- Keeps up with the industry development
- Many features built-in: Self-Monitoring, Container Registry, Docker CI Runner, ...
- Allows keeping CI/CD & code management in the same place
- Only CI/CD Tool
- Self-hosting is the only option
- All in one solution
- Self-Hosted or SaaS (managed)
- Using CI/CD without overhead of setting it up yourself
Git Branching Strategy
- A software Development workflow within the context of Git
- Describes how a development team will create, collaborate on, and merge branches of source code in a codebase
- Enable concurrent development in the codebase
Branching Strategy
Common Git Branching Strategies
- GitHub Flow
- Git Flow
- Gitlab Flow
Github Flow (Simplest Workflow)
Git Flow (Most Complex)
Gitlab Primary Branching Strategy
2 Variation:
- Environment Branches
- Release Branches
Gitlab Flow
Gitlab Flow (Environment Branches)
Gitlab Flow (Release Branches)
- Create Issue
- Create Production Branch
- Set/Add production branch as protected branch in setting
- Clone Project to your machine
- *** git pull
- Create and checkout feature branch
- Begin making modification to the codebase
- Add and Commit the changes
- Push the changes to Remote repository (Gitlab): git push -u origin <branch name>
- Create a merge request from feature branch to main branch
- Review the code changes in gitlab
- Approve Merge Request to Main Branch
- Create a merge request from main branch to production branch
- Create a git tag for versioning
- Source Codes assets is available in Deployment -> Releases
- Git pull main branch to sync remote repository to local repository
- Remove feature branch *if existed in local repo: git branch -d <feature branch name>
- git branch --all to check remote branch and local branch
- *if feature branch still exist in remote branch: git pull --prune
- Close the issue in gitlab
- With the continuous method of software development, you continuously build, test, and deploy iterative code changes.
- This iterative process helps reduce the chance that you develop new code based on buggy or failed previous versions.
- Strive to have less human intervention or even no intervention at all, from the development of new code until its deployment.
Gitlab CI/CD: Introduction
The three primary approaches for the continuous method :-
- Continuous Integration
- Continuous Delivery
- Continuous Deployment
Gitlab CI/CD: Introduction
- Continuous Integration
- Consider an application that has its code stored in a Git repository in GitLab. Developers push code changes every day, multiple times a day. For every push to the repository, you can create a set of scripts to build and test your application automatically. These scripts help decrease the chances that you introduce errors in your application.
- Each change submitted to an application, even to development branches, is built and tested automatically and continuously. These tests ensure the changes pass all tests, guidelines, and code compliance standards you established for your application.
Gitlab CI/CD: Introduction
2. Continuous Delivery
Not only is your application built and tested each time a code change is pushed to the codebase, the application is also deployed continuously. However, with continuous delivery, you trigger the deployments manually.
3. Continuous Deployment
Similar to Continuous Delivery. The difference is that instead of deploying your application manually, you set it to be deployed automatically. Human intervention is not required
Gitlab CI/CD: Introduction
Gitlab Architecture
Configuration File
-
.gitlab-ci.yml file
- Instructions for Gitlab CI/CD jobs
- Lives in the root of the repository
-
Gitlab Runner
- Lightweight agent that runs CI/CD jobs
- An Application that works with GitLab CI/CD to pick and execute CI/CD Jobs.
- Open-source and written in Go Language.
- You can add or remove runner into your GitLab architecture
- GitLab offer several shared runners available to every project in a GitLab instance.
- The GitLab runner can be installed on any platform or architecture that your own or manage.
Linux, macOS, Windows, FreeBSD, Cloud Provider, Bare Metal,Your work station and Docker - The GitLab runner can test any programming language:
.Net, Java, Python, C, PHP and others.
What is Gitlab Runner?
-
How do we build connection between gitlab runner server and gitlab instance server?
-
By register installed runner to gitlab server
-
Gitlab Runner
A runner can be:
Gitlab Runner
Shared VS Specific Runner
Tagged VS Untagged
Protected VS Unprotected
- Shell - Directly run commands as if writing them into terminal (bash or sh) or command prompt (cmd) or powershell.
- Docker - “main” machine scales up runners with *any* executor on demand Typical in cloud deployments.
- Docker - Execute inside of a docker image.
- Kubernetes - Runs as a pod in a K8s cluster. Can also feature auto scaling
- SSH - Similar to shell, but not as many features (bash only, no caching) Does allow you to SSH and execute commands on a machine you might not want to install runner on.
- Parallels - Parallels is a nice platform on top of VirtualBox.
- VirtualBox - Base VM for runner Main “main” creates a new VM for each needed runner
Runner Executor
Core Concept of Gitlab CI/CD
- Pipelines are the top-level component of continuous integration, delivery and deployment.
- Pipeline comprises of two things: Jobs and Stages
- Job defines what to do. Ex: A job test the code or to compile the code.
- Stages are collection of jobs. It define the order of the jobs.
- Pipeline are executed automatically and require no intervention once created.
- Gitlab also allows you to manually interact with a pipeline.
- Four usual stages of a CI/CD pipeline are test/build/stage/prod.
CI/CD Pipeline
CI/CD Pipeline
- To use CI/CD feature, there are two pre-requisite:
- Application code that host in Gitlab repository.
- .gitlab-ci.yml file that define your pipeline.
CI/CD Pipeline
- Jobs define what we want to accomplish in our pipeline.
- Executed by Runners
- Executed in Stages
- Stages define when and how to run jobs.
- Stages that run tests after stages that compile the code.
- Jobs in each stage are executed in parallel
- If all jobs in a stage succeed, the pipeline moves on to the next stage.
- if one job in a stage fails, the next stage is not (usually) executed
Jobs
Basic building block of pipeline
- Jobs are the most fundamental building block of a .gitlab-ci.yml file
job
job
job
job
Pipeline
.gitlab-ci.yml
- Basic building block of pipelines
- Job defined what to do
Jobs
- You can define arbitrary names for your job
- Must contain at least the script clause
- Script specify the command to execute
- You can define as many jobs as you want
before_script
is used to define a command that should be run before each job, including deploy jobs, but after the restoration of any artifacts
after_script
is used to define the command that will be run after each job, including failed ones.
Run before and after the script defined in each job
- Can update the image with the latest version of components
- They run within the job and can interact with the job
before_script and after_script
GitLab CI/CD pipeline configuration reference
- A job is defined as a list of keywords that define the job’s behavior.
- Configuration options for your Gitlab .gitlab-ci.yml file.
- The keywords available for jobs are:
https://docs.gitlab.com/ee/ci/yaml/
●image
●services
●script
●before_script & after_script
●variables
●Environment
●cache
●artifacts
●rules
●tags
●when
Stages
Group jobs
- You can group multiple jobs into stages that run in a defined order
- Multiple jobs in the same stage are executed in parallel
- Only if all jobs in a stage succeed, the pipeline moves on to the next stage
- If any job in a stage fails, the next stage is not executed and the pipeline ends
- Default Stages: Build, Test, Review, & Deploy
-
User can define custom stages & any number of jobs per stage
Stages seperate jobs into logical sections while Jobs perform the actual tasks
Stages
Needs
- Execute jobs in a certain order
- If it fails, it should skip the other dependent jobs
Relationship between jobs
Allow Failure
Keyword to continue the pipeline execution and report a success even if the job fails
"allow_failure:true"
Script
Inline shell commands and executing shell script
- Specify commands to execute
- Eg: OS commands, like Linux commands
Script
Jobs and Scripts
Only/Except
- By default, pipeline is triggered automatically for all branches
- Pipeline configuration applies to all git branches
- Only/Except: Job keywords to control when jobs are executed
- only: Define when a job runs
- except: Define when a job does not run
The rules syntax is an improved, more powerful solution for defining when jobs should run or not. Consider using rules instead of only/except to get the most out of your pipelines.
Only/Except
Basic Parameter
Predefined Variables
Predefined CI/CD variables are available in every Gitlab CI/CD pipeline
CI Tags
-
Tags are used to select a specific runner
- CI tags are different from Git tags
Workflow Rules
"workflow"
A global keyword, which configures the whole pipelines behavior
Workflow Rules
- Controls whether or not the whole pipeline is created
- "rules" can also be used on the job level as replacement for only/except
"workflow:rules"
Rules
Using variables to Determine What Triggered the Pipeline
Common Rules Use Cases
Rules
When is a Job created in Pipeline?
Rules Syntax
when Options
-
on_success
(default): Run the job only when no jobs in earlier stages fail or haveallow_failure: true
. -
on_failure
: Run the job only when at least one job in an earlier stage fails. A job in an earlier stage withallow_failure: true
is always considered successful. -
never
: Don’t run the job regardless of the status of jobs in earlier stages. Can only be used in arules
section orworkflow: rules
. -
always
: Run the job regardless of the status of jobs in earlier stages. Can also be used inworkflow:rules
. -
manual
: Run the job only when triggered manually. -
delayed
: Delay the execution of a job for a specified duration.
Rules
When is a Job NOT created in Pipeline?
Manual Execution
Rules Examples 1
Rules Examples 2
Rules Examples 3
Rules Examples 4
Variables
Variables Processing Order
- What is a template?
- The way to share CI/CD capabilities with other teams
- The way to consume CI/CD capabilities from other teams
- The way that GitLab engineering provides capabilities via templates
-
Templates are 0% magical, they are GitLab CI/CD YAML files
-
Templates are always executed into a CI/CD pipeline through an include statement in the project’s .gitlab-ci.yml
-
Template jobs are created in your CI/CD pipeline based on their defined stage (and any applicable rules)
Template
Template
To take advantage of SAST capabilities from GitLab engineering, we will use include:template
Template
Share Pipeline Definitions
Share Pipeline Definitions
Share Pipeline Definitions
Extending Jobs
Example Using extends
Artifacts
- Job artifacts are directories and files created by a job that you want to save after the job is finished; for example binaries that were built or reports generated by a test job.
- These artifacts are uploaded to GitLab and you can access them in a variety of ways
Environment
Preparation
Cache and Artifacts Download
Running
Scripts
Cache and Artifacts Upload
Cleanup
Gitlab job execution phases
Artifact Download
Example Build Artifact
Artifact VS Cache
Image
Environments
- Have this environment information in the pipeline
- Have an overview of which version deployed to which environment
- Link directly to the deployment endpoint
The environment keyword defines where the app is deployed and is defined by 3 parts.
When triggers jobs & stages manually (e.g. deploy to production)
Environments
Testing Tools
Testing
There are multiple type of testing that we can execute on stage Test
- Unit Test
- Code Quality Test
- Browser Performance Test
- Accessibility Test
- Load Performance Test
Unit Test
- Using your own language unit test.
-
There are four types of results:-
- Newly failed tests: Test cases which passed on base branch and failed on head branch
- Newly encountered errors: Test cases which passed on base branch and failed due to a test error on head branch
- Existing failures: Test cases which failed on base branch and failed on head branch
- Resolved failures: Test cases which failed on base branch and passed on head branch
●
●
Code Quality Test
- Gitlab uses plugins supported by Code Climate, which are free and open source. Code Quality does not require a Code Climate subscription.
- To ensure your project’s code stays simple, readable, and easy to contribute to
Static Application Security Test
(SAST)
Gitlab SAST use analyzer to check source code for any known vulnerabilities
include:
- template: Jobs/SAST.gitlab-ci.yml
#Static Application Security Test (SAST)
sast:
stage: test
artifacts:
paths: [gl-sast-report.json]
reports:
sast: gl-sast-report.json
Browser Performance Test
- GitLab uses Sitespeed.io, a free and open source tool, for measuring the rendering performance of web sites.
- The Sitespeed plugin that GitLab built outputs the performance score for each page analyzed in a file called browser-performance.json this data can be shown on Merge Requests.
Accessability Test
- GitLab integrates Pa11y into a CI job template.
- The a11y job analyzes a defined set of web pages and reports accessibility violations, warnings, and notices in a file named accessibility.
Load Performance Test
- GitLab uses k6, a free and open source tool, for measuring the system performance of applications under load.
- To perform various types of load tests against application endpoints such as APIs, Web Controllers, and so on.
Monitoring Tools
Application Performance Monitoring
Built on top of Elastic Stack to monitor application performance and services
CI/CD for Microservices
Monolithic Architecture
- All components are part of a single unit
- Everything is developed, deployed and scaled as 1 unit
- App must be written with 1 tech stack
- 1 single artifact, so you must redeploy the entire application on each update
Challenges of Monolithic Architecture
- Application is too large and complex
- You can only scale the entire app, instead of a specific service
- Difficulty if services need different dependency versions
- Release process take longer
- On every change, the entire application needs to be tested
- Entire application needs to be build and deployed
- Bug in any module can potentially bring down the entire application
Microservices
- Split your application into smaller, independent services
- Split based on business functionalities
- Services can be built and deployed separately
- Each microservice has its own version
Communication between Microservices
- Each services has its own API
- By calling the respective API endpoint, they can talk to each other
- Each microservice team can choose its own tech stack
- Each team can develop the service independently, without affecting others
- Added complexity just by the fact that a microservices application is distributed system
- Configure the communication between services
- More difficult to monitor with multiple instances of each service distributed across servers
Challenges of Microservices Architecture
Monorepo
MS
MS
MS
MS
MS
MS
Microservices
Monorepo
1 Git repository that contains many projects
Monorepo
- 1 Codebase with folders
- A directory for each service/project
- Makes the code management and development easier
- Clone and work only with 1 repo
- Changes can be tracked together, tested together and released together
- Share code and configurations
User
Thread
Post
Monorepo
Monorepo Challenges
- Tight coupling of projects
- Easier to break this criterion and develop more tightly coupled code
- Big source code, means git interactions becomes slow
- Additional logic necessary to make sure only service is build and deployed which had changes
Polyrepo
- 1 repository for each service
- Code is completely isolated
- Clone and work on them separately
- Gitlab Groups:
- Projects to configure connected project together (Groups)
- Helps to keep an overview
- Create shared secrets, CI/CD variables, Runners
- CI/CD: Own pipeline for each repository
User
Polyrepo
Thread
Post
Polyrepo Challenges
- Switching between projects tedious
- Searching, testing and debugging is more difficult
- Sharing resources more difficult
User
Thread
Post
Monorepo
User
Polyrepo
Thread
Post
For smaller microservice application
- Separate teams / better access control
- Have complete isolation
- Smaller code bases
- Own pipelines
Polyrepo VS Monorepo
Download Presentation Slide
Download Presentation Slide
THANK YOU
Gitlab CI/CD Training
By Abdullah Fathi
Gitlab CI/CD Training
- 749