Continuous Integration
持續整合
是一種軟體工程流程,在測試驅動開發(TDD)的作法中,通常還會搭配自動單元測試。
持續整合的提出主要是為解決軟體進行系統整合時面臨的各項問題。
Continuous Deployment/Delivery
持續部署/交付
持續部署是將開發的產品或系統以系統導向方式持續地將系統部署到指定階段環境
如開發的環境(development environment)
如上線環境(production environment)
持續部署是將開發的產品或是系統功能持續的改進與交付給需求方
自架版本
雲端服務版本
AWS CodePipeline
Azure DevOps
peter@devops-demo:~$ sudo apt-get install openjdk-11-jdk-headless
peter@devops-demo:~$ wget -q -O - https://pkg.jenkins.io/debian-stable/jenkins.io.key | sudo apt-key add -
[sudo] password for peter:
OK
peter@devops-demo:~$ sudo sh -c 'echo deb http://pkg.jenkins.io/debian-stable binary/ > /etc/apt/sources.list.d/jenkins.list'
peter@devops-demo:~$
peter@devops-demo:~$ sudo apt-get update
Hit:2 http://mirrors.digitalocean.com/ubuntu focal InRelease
Hit:3 https://repos-droplet.digitalocean.com/apt/droplet-agent main InRelease
Hit:4 http://mirrors.digitalocean.com/ubuntu focal-updates InRelease
Hit:5 http://mirrors.digitalocean.com/ubuntu focal-backports InRelease
Ign:1 https://pkg.jenkins.io/debian-stable binary/ InRelease
Get:6 https://pkg.jenkins.io/debian-stable binary/ Release [2044 B]
Get:7 https://pkg.jenkins.io/debian-stable binary/ Release.gpg [833 B]
Hit:8 http://security.ubuntu.com/ubuntu focal-security InRelease
Get:9 https://pkg.jenkins.io/debian-stable binary/ Packages [23.2 kB]
Fetched 26.1 kB in 1s (42.5 kB/s)
Reading package lists... Done
peter@devops-demo:~$ sudo apt-get install jenkins
......
peter@devops-demo:~$ systemctl status jenkins.service
● jenkins.service - Jenkins Continuous Integration Server
Loaded: loaded (/lib/systemd/system/jenkins.service; enabled; vendor preset: enabled)
Active: active (running) since Thu 2022-10-13 17:04:11 UTC; 20s ago
Main PID: 13531 (java)
Tasks: 51 (limit: 9508)
Memory: 2.2G
CGroup: /system.slice/jenkins.service
└─13531 /usr/bin/java -Djava.awt.headless=true -jar /usr/share/java/jenkins.war --webroot=/var>
peter@devops-demo:~$
root@devops-demo:/tmp# apt-get update
root@devops-demo:/tmp# apt-get install ca-certificates curl openssh-server postfix
root@devops-demo:~# cd /tmp/
root@devops-demo:/tmp# curl -LO https://packages.gitlab.com/install/repositories/gitlab/gitlab-ce/script.deb.sh
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 6865 100 6865 0 0 30923 0 --:--:-- --:--:-- --:--:-- 30923
root@devops-demo:/tmp# bash script.deb.sh
......
The repository is setup! You can now install packages.
root@devops-demo:/tmp# apt-get install gitlab-ce
.......
Thank you for installing GitLab!
GitLab was unable to detect a valid hostname for your instance.
Please configure a URL for your GitLab instance by setting `external_url`
configuration in /etc/gitlab/gitlab.rb file.
Then, you can start your GitLab instance by running the following command:
sudo gitlab-ctl reconfigure
For a comprehensive list of configuration options please see the Omnibus GitLab readme
https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/README.md
Help us improve the installation experience, let us know how we did with a 1 minute survey:
https://gitlab.fra1.qualtrics.com/jfe/form/SV_6kVqZANThUQ1bZb?installation=omnibus&release=15-4
root@devops-demo:/tmp#
root@devops-demo:/tmp# vim /etc/gitlab/gitlab.rb
root@devops-demo:/tmp# cat /etc/gitlab/gitlab.rb | grep '^external_url'
external_url 'https://gitlab.peterli.website'
root@devops-demo:/tmp# cat /etc/gitlab/gitlab.rb | grep '^letsencrypt'
letsencrypt['contact_emails'] = ['peter279k@gmail.com'] # This should be an array of email addresses to add as contacts
root@devops-demo:/tmp# gitlab-ctl reconfigure
......
Running handlers:
[2022-10-13T18:18:24+00:00] INFO: Running report handlers
Running handlers complete
[2022-10-13T18:18:24+00:00] INFO: Report handlers complete
Infra Phase complete, 650/1720 resources updated in 05 minutes 14 seconds
Notes:
Default admin account has been configured with following details:
Username: root
Password: You didn't opt-in to print initial root password to STDOUT.
Password stored to /etc/gitlab/initial_root_password. This file will be cleaned up in first reconfigure run after 24 hours.
NOTE: Because these credentials might be present in your log files in plain text, it is highly recommended to reset the password following https://docs.gitlab.com/ee/security/reset_user_password.html#reset-your-root-password.
gitlab Reconfigured!
root@devops-demo:/tmp# cat /etc/gitlab/initial_root_password
# WARNING: This value is valid only in the following conditions
# 1. If provided manually (either via `GITLAB_ROOT_PASSWORD` environment variable or via `gitlab_rails['initial_root_password']` setting in `gitlab.rb`, it was provided before database was seeded for the first time (usually, the first reconfigure run).
# 2. Password hasn't been changed manually, either via UI or via command line.
#
# If the password shown here doesn't work, you must reset the admin password following https://docs.gitlab.com/ee/security/reset_user_password.html#reset-your-root-password.
Password: l+ReQBz71nutZq7BPZ4nqD9eKpW472iWLk+30oHsfUI=
# NOTE: This file will be automatically deleted in the first reconfigure run after 24 hours.
root@devops-demo:/tmp#
root@devops-demo:/tmp# cd
root@devops-demo:~# mkdir hello-gitlab-ci
root@devops-demo:~# cd hello-gitlab-ci/
root@devops-demo:~/hello-gitlab-ci# vim .gitlab-ci.yml
root@devops-demo:~/hello-gitlab-ci# cat .gitlab-ci.yml
build-job:
stage: build
script:
- echo "Hello, $GITLAB_USER_LOGIN!"
test-job1:
stage: test
script:
- echo "This job tests something"
test-job2:
stage: test
script:
- echo "This job tests something, but takes more time than test-job1."
- echo "After the echo commands complete, it runs the sleep command for 20 seconds"
- echo "which simulates a test that runs 20 seconds longer than test-job1"
- sleep 20
deploy-prod:
stage: deploy
script:
- echo "This job deploys something from the $CI_COMMIT_BRANCH branch."
environment: production
root@devops-demo:~/hello-gitlab-ci# git init
Initialized empty Git repository in /root/hello-gitlab-ci/.git/
root@devops-demo:~/hello-gitlab-ci#
root@devops-demo:~/hello-gitlab-ci# git remote add origin https://gitlab.peterli.website/gitlab-instance-9f10964a/hello-gitlab-ci
root@devops-demo:~/hello-gitlab-ci# git config --global user.email peter279k@gmail.com
root@devops-demo:~/hello-gitlab-ci# git config --global user.name peter279k
root@devops-demo:~/hello-gitlab-ci# git add .
root@devops-demo:~/hello-gitlab-ci# git config --global core.editor vim
root@devops-demo:~/hello-gitlab-ci# git commit -a
[master (root-commit) fddb677] Initial commit
1 file changed, 23 insertions(+)
create mode 100644 .gitlab-ci.yml
root@devops-demo:~/hello-gitlab-ci# git push origin master
Username for 'https://gitlab.peterli.website': root
Password for 'https://root@gitlab.peterli.website':
warning: redirecting to https://gitlab.peterli.website/gitlab-instance-9f10964a/hello-gitlab-ci.git/
Enumerating objects: 3, done.
Counting objects: 100% (3/3), done.
Delta compression using up to 4 threads
Compressing objects: 100% (2/2), done.
Writing objects: 100% (3/3), 496 bytes | 496.00 KiB/s, done.
Total 3 (delta 0), reused 0 (delta 0)
To https://gitlab.peterli.website/gitlab-instance-9f10964a/hello-gitlab-ci
* [new branch] master -> master
root@devops-demo:~/hello-gitlab-ci#
root@devops-demo:~# curl -LJO "https://gitlab-runner-downloads.s3.amazonaws.com/latest/deb/gitlab-runner_amd64.deb"
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
48 414M 48 203M 0 0 10.6M 0 0:00:38 0:00:19 0:00:19 12.1M
root@devops-demo:~# dpkg -i gitlab-runner_amd64.deb
Selecting previously unselected package gitlab-runner.
(Reading database ... 157514 files and directories currently installed.)
Preparing to unpack gitlab-runner_amd64.deb ...
Unpacking gitlab-runner (15.4.0) ...
......
15.4.0
Check and remove all unused containers (both dangling and unreferenced) including volumes.
------------------------------------------------------------------------------------------
Total reclaimed space: 0B
root@devops-demo:~# gitlab-runner status
Runtime platform arch=amd64 os=linux pid=23296 revision=43b2dc3d version=15.4.0
gitlab-runner: Service is running
root@devops-demo:~/hello-gitlab-ci# gitlab-runner register
Runtime platform arch=amd64 os=linux pid=24415 revision=43b2dc3d version=15.4.0
Running in system-mode.
Enter the GitLab instance URL (for example, https://gitlab.com/):
https://gitlab.peterli.website/
Enter the registration token:
GR1348941Z1qeA8HkLxPvTaji52Zp
Enter a description for the runner:
[devops-demo]: hello-gitlab-ci
Enter tags for the runner (comma-separated):
hello-gitlab-ci
Enter optional maintenance note for the runner:
Registering runner... succeeded runner=GR1348941Z1qeA8Hk
Enter an executor: docker, docker-ssh, shell, docker+machine, docker-ssh+machine, kubernetes, custom, ssh, virtualbox, parallels:
docker
Enter the default Docker image (for example, ruby:2.7):
python:3.7
Runner registered successfully. Feel free to start it, but if it's running already the config should be automatically reloaded!
Configuration (with the authentication token) was saved in "/etc/gitlab-runner/config.toml"
root@devops-demo:~/hello-gitlab-ci#
language: python
python:
- "2.7"
- "3.4"
- "3.5"
- "3.6" # current default Python on Travis CI
- "3.7"
- "3.8"
- "3.9"
- "3.9-dev" # 3.9 development branch
- "nightly" # nightly build
# command to install dependencies
install:
- pip install -r requirements.txt
# command to run tests
script:
- pytest
version: 2.1
orbs:
python: circleci/python@2.0.3
heroku: circleci/heroku@1.2.6
jobs:
build_and_test: # this can be any name you choose
executor: python/default
steps:
- checkout
- python/install-packages:
pkg-manager: pip
- run:
name: Run tests
command: python -m pytest
- persist_to_workspace:
root: ~/project
paths:
- .
deploy: # this can be any name you choose
executor: python/default
steps:
- attach_workspace:
at: ~/project
- heroku/deploy-via-git:
force: true # force push when pushing to the heroku remote, see: https://devcenter.heroku.com/articles/git
workflows:
test_my_app:
jobs:
- build_and_test
- deploy:
requires:
- build_and_test # only deploy if the build_and_test job has completed
filters:
branches:
only: main # only deploy when on main
name: Test Workflow
on: push
jobs:
first-job:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v2
- name: Run Action
# we can give directory path that contains action.yaml or repo address in username/repository_name@version format
# in our case it would be nashmaniac/create-issue-action@v1.0 where v1.0 is the version of action
# for now we will give directory path. The directory must contain action.yaml
uses: ./
# pass user input as arguments
with:
title: Build Failed
token: ${{secrets.GITHUB_TOKEN}} # default token in GitHub Workflow
assignees: ${{github.actor}} # actor who started the workflow
labels: worflow-failed
body: Workflow failed for commit ${{github.sha}}
CI/CD工具或服務名稱 | 優點 | 缺點 |
---|---|---|
Travis CI | 快速、方便與好用 | 只有提供雲端服務 |
GitHub Workflow | 與GitHub平台整合佳 | 只有提供雲端服務 |
CircleCI | 方便與好用 | 只有提供雲端服務 |
GitLab CI/CD | 與GitLab平台整合佳 | 安裝自架版本後會附加GitLab的專案託管平台 |
Jenknis | 可自架、維護與管理 | 需要會寫平台自定義Pipelines語言 或是安裝外掛支援YAML檔案編寫CI/CD設定檔 有很多既有的功能是支援Java專案的 其他程式語言專案的測試環境要先自己架好 也可以使用Docker image建置測試環境 |
https://www.digitalocean.com/community/tutorials/how-to-install-jenkins-on-ubuntu-20-04
https://www.jenkins.io/blog/2018/04/25/configuring-jenkins-pipeline-with-yaml-file
https://github.com/jenkinsci/pipeline-as-yaml-plugin/blob/master/src/test/resources/job/pipelineTestWithBranches.yml
https://www.digitalocean.com/community/tutorials/how-to-install-and-configure-gitlab-on-ubuntu-18-04
https://docs.gitlab.com/runner/install/linux-manually.html
https://docs.docker.com/engine/install/ubuntu
https://docs.gitlab.com/ee/ci/examples
https://docs.gitlab.com/ee/ci/quick_start
https://docs.travis-ci.com/user/languages/python
https://circleci.com/docs/language-python
https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-python
https://medium.com/@xthewiz/notify-jenkins-for-push-event-on-gitlab-18e2e4104827
https://www.cnblogs.com/panzi/p/10536654.html