CI/CD工具介紹
2022/10/13
名詞解釋
-
CI
-
Continuous Integration
-
持續整合
-
是一種軟體工程流程,在測試驅動開發(TDD)的作法中,通常還會搭配自動單元測試。
-
持續整合的提出主要是為解決軟體進行系統整合時面臨的各項問題。
-
-
CD
-
Continuous Deployment/Delivery
-
持續部署/交付
-
持續部署是將開發的產品或系統以系統導向方式持續地將系統部署到指定階段環境
-
如開發的環境(development environment)
-
如上線環境(production environment)
-
-
持續部署是將開發的產品或是系統功能持續的改進與交付給需求方
-
CI工具分類
-
由雲端主機供應商推出
-
私人公司推出的CI/CD工具產品或服務
-
自架版本
-
雲端服務版本
-
由雲端主機供應商推出
-
AWS
-
AWS CodePipeline
-
-
Azure
-
Azure DevOps
-
-
Google Cloud Platform
- Cloud Build、Cloud Deploy 以及 Cloud Run
私人公司推出的CI/CD工具產品或服務
-
Jenkins
-
GitLab CI/CD
自架版本(self-hosted)
私人公司推出的CI/CD工具產品或服務
-
Travis CI
-
GitLab.com上的GitLab CI/CD
- CircleCI
- GitHub Workflows
雲端服務版本(cloud)
Jenkins安裝(自架需要先安裝)
peter@devops-demo:~$ sudo apt-get install openjdk-11-jdk-headless
peter@devops-demo:~$ wget -q -O - https://pkg.jenkins.io/debian-stable/jenkins.io.key | sudo apt-key add -
[sudo] password for peter:
OK
peter@devops-demo:~$ sudo sh -c 'echo deb http://pkg.jenkins.io/debian-stable binary/ > /etc/apt/sources.list.d/jenkins.list'
peter@devops-demo:~$
peter@devops-demo:~$ sudo apt-get update
Hit:2 http://mirrors.digitalocean.com/ubuntu focal InRelease
Hit:3 https://repos-droplet.digitalocean.com/apt/droplet-agent main InRelease
Hit:4 http://mirrors.digitalocean.com/ubuntu focal-updates InRelease
Hit:5 http://mirrors.digitalocean.com/ubuntu focal-backports InRelease
Ign:1 https://pkg.jenkins.io/debian-stable binary/ InRelease
Get:6 https://pkg.jenkins.io/debian-stable binary/ Release [2044 B]
Get:7 https://pkg.jenkins.io/debian-stable binary/ Release.gpg [833 B]
Hit:8 http://security.ubuntu.com/ubuntu focal-security InRelease
Get:9 https://pkg.jenkins.io/debian-stable binary/ Packages [23.2 kB]
Fetched 26.1 kB in 1s (42.5 kB/s)
Reading package lists... Done
peter@devops-demo:~$ sudo apt-get install jenkins
......
peter@devops-demo:~$ systemctl status jenkins.service
● jenkins.service - Jenkins Continuous Integration Server
Loaded: loaded (/lib/systemd/system/jenkins.service; enabled; vendor preset: enabled)
Active: active (running) since Thu 2022-10-13 17:04:11 UTC; 20s ago
Main PID: 13531 (java)
Tasks: 51 (limit: 9508)
Memory: 2.2G
CGroup: /system.slice/jenkins.service
└─13531 /usr/bin/java -Djava.awt.headless=true -jar /usr/share/java/jenkins.war --webroot=/var>
peter@devops-demo:~$
Jenkins設定並選擇安裝推薦的外掛
Jenkins設定並選擇安裝推薦的外掛
Jenkins建立第一個管理員使用者
Jenkins執行instance設定
開始使用Jenkins
開始使用Jenkins
在Jenkins上建立新的作業
在Jenkins上建立新的作業-Hello Pipelines
在Jenkins上建立新的作業-執行Hello Pipelines
在Jenkins上建立新的作業-執行Hello Pipelines
安裝Pipeline As YAML外掛
安裝Pipeline As YAML外掛
安裝Pipeline As YAML外掛
使用Pipeline As YAML外掛
執行使用Pipeline As YAML外掛的pipelines
GitLab CI/CD-安裝GitLab CE版本
root@devops-demo:/tmp# apt-get update
root@devops-demo:/tmp# apt-get install ca-certificates curl openssh-server postfix
root@devops-demo:~# cd /tmp/
root@devops-demo:/tmp# curl -LO https://packages.gitlab.com/install/repositories/gitlab/gitlab-ce/script.deb.sh
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 6865 100 6865 0 0 30923 0 --:--:-- --:--:-- --:--:-- 30923
root@devops-demo:/tmp# bash script.deb.sh
......
The repository is setup! You can now install packages.
root@devops-demo:/tmp# apt-get install gitlab-ce
.......
Thank you for installing GitLab!
GitLab was unable to detect a valid hostname for your instance.
Please configure a URL for your GitLab instance by setting `external_url`
configuration in /etc/gitlab/gitlab.rb file.
Then, you can start your GitLab instance by running the following command:
sudo gitlab-ctl reconfigure
For a comprehensive list of configuration options please see the Omnibus GitLab readme
https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/README.md
Help us improve the installation experience, let us know how we did with a 1 minute survey:
https://gitlab.fra1.qualtrics.com/jfe/form/SV_6kVqZANThUQ1bZb?installation=omnibus&release=15-4
root@devops-demo:/tmp#
GitLab CI/CD-安裝GitLab CE版本(設定GitLab)
root@devops-demo:/tmp# vim /etc/gitlab/gitlab.rb
root@devops-demo:/tmp# cat /etc/gitlab/gitlab.rb | grep '^external_url'
external_url 'https://gitlab.peterli.website'
root@devops-demo:/tmp# cat /etc/gitlab/gitlab.rb | grep '^letsencrypt'
letsencrypt['contact_emails'] = ['peter279k@gmail.com'] # This should be an array of email addresses to add as contacts
root@devops-demo:/tmp# gitlab-ctl reconfigure
......
Running handlers:
[2022-10-13T18:18:24+00:00] INFO: Running report handlers
Running handlers complete
[2022-10-13T18:18:24+00:00] INFO: Report handlers complete
Infra Phase complete, 650/1720 resources updated in 05 minutes 14 seconds
Notes:
Default admin account has been configured with following details:
Username: root
Password: You didn't opt-in to print initial root password to STDOUT.
Password stored to /etc/gitlab/initial_root_password. This file will be cleaned up in first reconfigure run after 24 hours.
NOTE: Because these credentials might be present in your log files in plain text, it is highly recommended to reset the password following https://docs.gitlab.com/ee/security/reset_user_password.html#reset-your-root-password.
gitlab Reconfigured!
root@devops-demo:/tmp# cat /etc/gitlab/initial_root_password
# WARNING: This value is valid only in the following conditions
# 1. If provided manually (either via `GITLAB_ROOT_PASSWORD` environment variable or via `gitlab_rails['initial_root_password']` setting in `gitlab.rb`, it was provided before database was seeded for the first time (usually, the first reconfigure run).
# 2. Password hasn't been changed manually, either via UI or via command line.
#
# If the password shown here doesn't work, you must reset the admin password following https://docs.gitlab.com/ee/security/reset_user_password.html#reset-your-root-password.
Password: l+ReQBz71nutZq7BPZ4nqD9eKpW472iWLk+30oHsfUI=
# NOTE: This file will be automatically deleted in the first reconfigure run after 24 hours.
root@devops-demo:/tmp#
GitLab CI/CD-瀏覽GitLab網址(https://gitlab.peterli.website)
GitLab CI/CD-瀏覽GitLab網址(建立專案)
GitLab CI/CD-瀏覽GitLab網址(建立Git專案)
root@devops-demo:/tmp# cd
root@devops-demo:~# mkdir hello-gitlab-ci
root@devops-demo:~# cd hello-gitlab-ci/
root@devops-demo:~/hello-gitlab-ci# vim .gitlab-ci.yml
root@devops-demo:~/hello-gitlab-ci# cat .gitlab-ci.yml
build-job:
stage: build
script:
- echo "Hello, $GITLAB_USER_LOGIN!"
test-job1:
stage: test
script:
- echo "This job tests something"
test-job2:
stage: test
script:
- echo "This job tests something, but takes more time than test-job1."
- echo "After the echo commands complete, it runs the sleep command for 20 seconds"
- echo "which simulates a test that runs 20 seconds longer than test-job1"
- sleep 20
deploy-prod:
stage: deploy
script:
- echo "This job deploys something from the $CI_COMMIT_BRANCH branch."
environment: production
root@devops-demo:~/hello-gitlab-ci# git init
Initialized empty Git repository in /root/hello-gitlab-ci/.git/
root@devops-demo:~/hello-gitlab-ci#
GitLab CI/CD-瀏覽GitLab網址(建立Git專案)
root@devops-demo:~/hello-gitlab-ci# git remote add origin https://gitlab.peterli.website/gitlab-instance-9f10964a/hello-gitlab-ci
root@devops-demo:~/hello-gitlab-ci# git config --global user.email peter279k@gmail.com
root@devops-demo:~/hello-gitlab-ci# git config --global user.name peter279k
root@devops-demo:~/hello-gitlab-ci# git add .
root@devops-demo:~/hello-gitlab-ci# git config --global core.editor vim
root@devops-demo:~/hello-gitlab-ci# git commit -a
[master (root-commit) fddb677] Initial commit
1 file changed, 23 insertions(+)
create mode 100644 .gitlab-ci.yml
root@devops-demo:~/hello-gitlab-ci# git push origin master
Username for 'https://gitlab.peterli.website': root
Password for 'https://root@gitlab.peterli.website':
warning: redirecting to https://gitlab.peterli.website/gitlab-instance-9f10964a/hello-gitlab-ci.git/
Enumerating objects: 3, done.
Counting objects: 100% (3/3), done.
Delta compression using up to 4 threads
Compressing objects: 100% (2/2), done.
Writing objects: 100% (3/3), 496 bytes | 496.00 KiB/s, done.
Total 3 (delta 0), reused 0 (delta 0)
To https://gitlab.peterli.website/gitlab-instance-9f10964a/hello-gitlab-ci
* [new branch] master -> master
root@devops-demo:~/hello-gitlab-ci#
GitLab CI/CD-瀏覽GitLab網址(運行專案之GitLab CI/CD)
GitLab CI/CD-瀏覽GitLab網址(安裝Docker)
GitLab CI/CD-瀏覽GitLab網址(安裝GitLab Runner)
root@devops-demo:~# curl -LJO "https://gitlab-runner-downloads.s3.amazonaws.com/latest/deb/gitlab-runner_amd64.deb"
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
48 414M 48 203M 0 0 10.6M 0 0:00:38 0:00:19 0:00:19 12.1M
root@devops-demo:~# dpkg -i gitlab-runner_amd64.deb
Selecting previously unselected package gitlab-runner.
(Reading database ... 157514 files and directories currently installed.)
Preparing to unpack gitlab-runner_amd64.deb ...
Unpacking gitlab-runner (15.4.0) ...
......
15.4.0
Check and remove all unused containers (both dangling and unreferenced) including volumes.
------------------------------------------------------------------------------------------
Total reclaimed space: 0B
root@devops-demo:~# gitlab-runner status
Runtime platform arch=amd64 os=linux pid=23296 revision=43b2dc3d version=15.4.0
gitlab-runner: Service is running
GitLab CI/CD-瀏覽GitLab網址(安裝GitLab Runner)
root@devops-demo:~/hello-gitlab-ci# gitlab-runner register
Runtime platform arch=amd64 os=linux pid=24415 revision=43b2dc3d version=15.4.0
Running in system-mode.
Enter the GitLab instance URL (for example, https://gitlab.com/):
https://gitlab.peterli.website/
Enter the registration token:
GR1348941Z1qeA8HkLxPvTaji52Zp
Enter a description for the runner:
[devops-demo]: hello-gitlab-ci
Enter tags for the runner (comma-separated):
hello-gitlab-ci
Enter optional maintenance note for the runner:
Registering runner... succeeded runner=GR1348941Z1qeA8Hk
Enter an executor: docker, docker-ssh, shell, docker+machine, docker-ssh+machine, kubernetes, custom, ssh, virtualbox, parallels:
docker
Enter the default Docker image (for example, ruby:2.7):
python:3.7
Runner registered successfully. Feel free to start it, but if it's running already the config should be automatically reloaded!
Configuration (with the authentication token) was saved in "/etc/gitlab-runner/config.toml"
root@devops-demo:~/hello-gitlab-ci#
GitLab CI/CD-瀏覽GitLab網址(安裝GitLab Runner)
Travis CI(雲端服務版本)
.travis.yml
language: python
python:
- "2.7"
- "3.4"
- "3.5"
- "3.6" # current default Python on Travis CI
- "3.7"
- "3.8"
- "3.9"
- "3.9-dev" # 3.9 development branch
- "nightly" # nightly build
# command to install dependencies
install:
- pip install -r requirements.txt
# command to run tests
script:
- pytest
Circle CI(雲端服務版本)
.circleci/config.yml
version: 2.1
orbs:
python: circleci/python@2.0.3
heroku: circleci/heroku@1.2.6
jobs:
build_and_test: # this can be any name you choose
executor: python/default
steps:
- checkout
- python/install-packages:
pkg-manager: pip
- run:
name: Run tests
command: python -m pytest
- persist_to_workspace:
root: ~/project
paths:
- .
deploy: # this can be any name you choose
executor: python/default
steps:
- attach_workspace:
at: ~/project
- heroku/deploy-via-git:
force: true # force push when pushing to the heroku remote, see: https://devcenter.heroku.com/articles/git
workflows:
test_my_app:
jobs:
- build_and_test
- deploy:
requires:
- build_and_test # only deploy if the build_and_test job has completed
filters:
branches:
only: main # only deploy when on main
GitHub Workflow(雲端服務版本)
.github/workflows/your-ci-name.yml
name: Test Workflow
on: push
jobs:
first-job:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v2
- name: Run Action
# we can give directory path that contains action.yaml or repo address in username/repository_name@version format
# in our case it would be nashmaniac/create-issue-action@v1.0 where v1.0 is the version of action
# for now we will give directory path. The directory must contain action.yaml
uses: ./
# pass user input as arguments
with:
title: Build Failed
token: ${{secrets.GITHUB_TOKEN}} # default token in GitHub Workflow
assignees: ${{github.actor}} # actor who started the workflow
labels: worflow-failed
body: Workflow failed for commit ${{github.sha}}
總結比較
CI/CD工具或服務名稱 | 優點 | 缺點 |
---|---|---|
Travis CI | 快速、方便與好用 | 只有提供雲端服務 |
GitHub Workflow | 與GitHub平台整合佳 | 只有提供雲端服務 |
CircleCI | 方便與好用 | 只有提供雲端服務 |
GitLab CI/CD | 與GitLab平台整合佳 | 安裝自架版本後會附加GitLab的專案託管平台 |
Jenknis | 可自架、維護與管理 | 需要會寫平台自定義Pipelines語言 或是安裝外掛支援YAML檔案編寫CI/CD設定檔 有很多既有的功能是支援Java專案的 其他程式語言專案的測試環境要先自己架好 也可以使用Docker image建置測試環境 |
參考資料
-
https://www.digitalocean.com/community/tutorials/how-to-install-jenkins-on-ubuntu-20-04
-
https://www.jenkins.io/blog/2018/04/25/configuring-jenkins-pipeline-with-yaml-file
-
https://github.com/jenkinsci/pipeline-as-yaml-plugin/blob/master/src/test/resources/job/pipelineTestWithBranches.yml
-
https://www.digitalocean.com/community/tutorials/how-to-install-and-configure-gitlab-on-ubuntu-18-04
-
https://docs.gitlab.com/runner/install/linux-manually.html
-
https://docs.docker.com/engine/install/ubuntu
-
https://docs.gitlab.com/ee/ci/examples
-
https://docs.gitlab.com/ee/ci/quick_start
-
https://docs.travis-ci.com/user/languages/python
-
https://circleci.com/docs/language-python
-
https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-python
-
https://medium.com/@xthewiz/notify-jenkins-for-push-event-on-gitlab-18e2e4104827
-
https://www.cnblogs.com/panzi/p/10536654.html
CI/CD Tools Survey
By peter279k
CI/CD Tools Survey
CI/CD Tools Survey
- 462