CICD on OpenShift
Poznań 14-11-2019 @ <CONSDATA.TECH>
Andrzej Goławski
Application Delivery Cycle
Dev
Code
GIT
Build
Artifact
Deploy Test
Deploy Blue
Testing
Internal Testing
Switch Traffic to Blue
Monitoring
Deploy Green
Switch Traffic to Green
Application Delivery Cycle
GIT
Code
Build
Deploy
APP
APP
APP
inner registry
push
pull
pull
clone
app
app-test
app-prod
Deploy
Deploy
Build Process on OpenShift
openshift
app
inner registry
s2i-java11-maven
bc
s2i
application
is
is
GIT
push
push
clone
oc new-build --name=app s2i-java11-maven~http://URL-TO-REPO
oc new-project app
Build Process on OpenShift
#!/bin/bash
#Create app project
oc new-project app
#Create Build Configuration
oc new-build --name=app s2i-java11-maven~$APP_GIT_URL -n app
Build Application script: build-app.sh
Deploy Process on OpenShift
app
app-test
inner registry
bc
is
app:latest
dc
rc
pod
pull
app:test
oc new-app --name=app-test --image-stream=app/app:test --allow-missing-imagestream-tags
oc tag app:latest app:test -n app
oc new-project app-test
oc policy add-role-to-user system:image-puller system:serviceaccount:app-test:default -n app
Deploy Process on OpenShift
#!/bin/bash
#Create app-test project
oc new-project app-test
#Add permissions to pull image from app project
oc adm policy add-role-to-user system:image-puller \
system:serviceaccount:app-test:default -n app
#Create DeploymentConfiguration
oc new-app --name=app-test --image-stream=app/app:test --allow-missing-imagestream-tags
#Remove unnecessary Image Stream
oc delete is app-test -n app-test
#Add readiness probe
oc set probe dc/app-test --readiness --get-url=http://:8080/actuator/health -n app-test
Deploy Application on Test script: create-app-test.sh
Network on OpenShift
app-test
pod
svc
route
http request
oc expose dc/app-test
oc expose scv/app-test
Deploy Production
Deploy Production
#!/bin/bash
#Create project prod
oc new-project app-prod
#Add permissions to pull image from app project
oc policy add-role-to-user system:image-puller \
system:serviceaccount:app-prod:default -n app
#Create blue and green deployments
oc new-app --name=app-blue --image-stream=app/app:prod-blue \
--allow-missing-imagestream-tags -n app-prod
oc new-app --name=app-green --image-stream=app/app:prod-green \
--allow-missing-imagestream-tags -n app-prod
#Remove unnecessary Image Streams
oc delete is --all -n app-prod
#Expose prod applications
oc expose deploymentconfig.apps.openshift.io/app-blue --port=8080 -n app-prod
oc expose deploymentconfig.apps.openshift.io/app-green --port=8080 -n app-prod
oc expose service app-blue --hostname=blue.apps-crc.testing -n app-prod
oc expose service app-green --hostname=green.apps-crc.testing -n app-prod
#Add readiness probe
oc set probe dc/app-blue --readiness --get-url=http://:8080/actuator/health -n app-prod
oc set probe dc/app-green --readiness --get-url=http://:8080/actuator/health -n app-prod
Deploy Production script: create-app-prod.sh
Split traffic
app-test
pod
svc
route
http request
pod
svc
70%
30%
oc expose svc/app-green
oc set route-backends app-green app-green=70 app-blue=30
Expose Production
#!/bin/bash
#Create production Route
oc expose service app-green --name prod \
--hostname=prod.apps-crc.testing -n app-prod
oc set route-backends prod app-green=100 app-blue=0 -n app-prod
Expose production script: expose-prod.sh
"CICD" on OpenShift without Jenkis
oc start-build app -n app
build project
oc tag app:latest app:test -n app
deploy test
oc tag app:test app:prod-blue -n app
deploy blue
oc tag app:test app:prod-green -n app
switch traffic to blue
oc set route-backends prod \
app-green=0 app-blue=100 -n app-prod
deploy green
oc set route-backends prod \
app-green=100 app-blue=0 -n app-prod
switch traffic to green
Jenkins on OpenShift
Install Jenkins on OpenShift
Install using CLI
oc new-app jenkins-persistent
oc get templates -n openshift | grep jenkins
jenkins-ephemeral Jenkins service, without persistent storage....
jenkins-persistent Jenkins service, with persistent storage....
Available Templates
Install using Web Console
Jenkins on OpenShift
cicd
J
bc
kind: BuildConfig
apiVersion: v1
metadata:
name: app-pipeline
namespace: cicd
labels:
name: app-pipeline
spec:
strategy:
type: JenkinsPipeline
jenkinsPipelineStrategy:
jenkinsfile: |-
echo 'hello world!'
oc start-build app-pipeline -n cicd
oc apply -f app-pipeline.yaml -n cicd
Jenkins on OpenShift
cicd
J
bc
stage('build') {
echo 'build'
}
stage('deploy') {
echo 'deploy'
}
jenkinsfile
oc start-build app-pipeline -n cicd
Jenkins on OpenShift
cicd
J
bc
node {
stage('build') {
echo 'build'
}
stage('deploy') {
echo 'deploy'
}
}
jenkinsfile
oc start-build app-pipeline -n cicd
node('slave') {
stage('build') {
echo 'build'
}
stage('deploy') {
echo 'deploy'
}
}
slave
execute build
execute deploy
Jenkins on OpenShift
cicd
J
bc
node {
stage('build') {
sh 'oc start-build app -n app'
}
stage('deploy test') {
sh 'oc tag app:latest app:test -n app'
sh 'oc rollout latest app-test -n app-test'
}
}
jenkinsfile
oc start-build app-pipeline -n cicd
app
app-test
build
deploy
dc
bc
OpenShift Jenkins Pipeline (DSL) Plugin
OpenShift Jenkins Pipeline (DSL) Plugin
Build Configuration Definition
kind: BuildConfig
apiVersion: v1
metadata:
name: app-pipeline
namespace: cicd
labels:
name: app-pipeline
spec:
strategy:
type: JenkinsPipeline
jenkinsPipelineStrategy:
jenkinsfile: |-
node {
stage("build") {
.....
}
}
OpenShift Jenkins Pipeline (DSL) Plugin
Build
node {
stage("build") {
openshift.withCluster() {
openshift.withProject('app') {
def buildConfig = openshift.selector("bc", "app")
def build = buildConfig.startBuild("--incremental")
build.untilEach() {
return it.object().status.phase == "Complete"
}
}
}
}
}
OpenShift 3.x
node {
stage("build") {
openshiftBuild bldCfg: 'app', namespace: 'app'
}
}
OpenShift Jenkins Pipeline (DSL) Plugin
Deploy Test
node {
stage("build") {.....}
stage("deploy to test") {
openshift.withCluster() {
openshift.withProject('app-test') {
openshift.tag( 'app/app:latest', 'app/app:test')
def deployConfig = openshift.selector("dc", "app-test")
def replicasNumber = deployConfig.object().spec.replicas
deployConfig.rollout().latest()
def latestDeploymentVersion =
deployConfig.object().status.latestVersion
def replicationController =
openshift.selector('rc', "app-test-${latestDeploymentVersion}")
replicationController.untilEach() {
def replicationControllerObject = it.object()
return (replicasNumber.equals(
replicationControllerObject.status.readyReplicas))
}
}
}
}
}
OpenShift Jenkins Pipeline (DSL) Plugin
node {
stage("build") {.....}
stage("deploy to test") {
openshiftDeploy depCfg: 'app-test', namespace: 'app-test'
}
}
OpenShift 3.x
OpenShift Jenkins Pipeline (DSL) Plugin
Extract Deploy to method
node {
stage("build") {.....}
stage("deploy to test") {
openshift.withCluster() {
openshift.withProject('app-test') {
openshift.tag( 'app/app:latest', 'app/app:test')
deploy("app-test")
}
}
}
}
def deploy(String deploymentConfigurationName) {
def deployConfig = openshift.selector("dc", deploymentConfigurationName)
def replicasNumber = deployConfig.object().spec.replicas
deployConfig.rollout().latest()
def latestDeploymentVersion = deployConfig.object().status.latestVersion
def replicationController =
openshift.selector('rc', "${deploymentConfigurationName}-${latestDeploymentVersion}")
replicationController.untilEach() {
def replicationControllerObject = it.object()
return (replicasNumber.equals(
replicationControllerObject.status.readyReplicas))
}
}
OpenShift Jenkins Pipeline (DSL) Plugin
Testing
node {
stage("build") {.....}
stage("deploy to test") {.....}
stage("manual testing") {
input message: 'Should deploy to production as blue?', ok: 'Yes'
}
}
def deploy(String deploymentConfigurationName) {.....}
OpenShift Jenkins Pipeline (DSL) Plugin
Deploy Blue
node {
stage("build") {.....}
stage("deploy to test") {.....}
stage("manual testing") {.....}
stage("deploy blue on production") {
openshift.withCluster() {
openshift.withProject('app-prod') {
openshift.tag('app/app:latest', 'app/app:prod-blue')
deploy("app-blue")
}
}
}
}
def deploy(String deploymentConfigurationName) {.....}
OpenShift Jenkins Pipeline (DSL) Plugin
Internal testing Blue
node {
stage("build") {.....}
stage("deploy to test") {.....}
stage("manual testing") {.....}
stage("deploy blue on production") {.....}
stage("internal testing blue") {
input message: 'Should swich traffic to blue?', ok: 'Yes'
}
}
def deploy(String deploymentConfigurationName) {.....}
OpenShift Jenkins Pipeline (DSL) Plugin
Switch traffic to Blue
node {
stage("build") {.....}
stage("deploy to test") {.....}
stage("manual testing") {.....}
stage("deploy on production as blue") {.....}
stage("internal testing blue") {.....}
stage("swich traffic to blue") {
openshift.withCluster() {
openshift.withProject('app-prod') {
def routeObject = openshift.selector("route", "prod").object()
routeObject.spec.alternateBackends[0].weight=100
routeObject.spec.to.weight=0
openshift.apply(routeObject)
}
}
}
}
def deploy(String deploymentConfigurationName) {.....}
or
stage("swich traffic to blue") {
sh 'oc set route-backends prod app-green=0 app-blue=100 -n app-prod'
}
OpenShift Jenkins Pipeline (DSL) Plugin
Extract switch traffic to method
node {
stage("build") {.....}
stage("deploy to test") {.....}
stage("manual testing") {.....}
stage("deploy on production as blue") {.....}
stage("internal testing blue") {.....}
stage("swich traffic to blue") {
openshift.withCluster() {
openshift.withProject('app-prod') {
switchTraffic('prod', 0, 100)
}
}
}
def deploy(String deploymentConfigurationName) {.....}
def switchTraffic(String routeName, int greenWeight, int blueWeight) {
def routeObject = openshift.selector("route", routeName).object()
routeObject.spec.alternateBackends[0].weight=blueWeight
routeObject.spec.to.weight=greenWeight
openshift.apply(routeObject)
}
OpenShift Jenkins Pipeline (DSL) Plugin
Testing Blue on production
node {
stage("build") {.....}
stage("deploy to test") {.....}
stage("manual testing") {.....}
stage("deploy blue on production") {.....}
stage("internal testing blue") {.....}
stage("swich traffic to blue") {.....}
stage("testing blue on production") {
input message: 'Should deploy to production as blue?', ok: 'Yes'
}
}
def deploy(String deploymentConfigurationName) {.....}
def switchTraffic(String routeName, int greenWeight, int blueWeight) {....}
OpenShift Jenkins Pipeline (DSL) Plugin
Deploy Green
node {
stage("build") {.....}
stage("deploy to test") {.....}
stage("manual testing") {.....}
stage("deploy blue on production") {.....}
stage("internal testing blue") {.....}
stage("swich traffic to blue") {.....}
stage("testing blue on production") {.....}
stage("deploy green on production") {
openshift.withCluster() {
openshift.withProject('app-prod') {
openshift.tag('app/app:latest', 'app/app:prod-green')
deploy("app-green")
}
}
}
}
def deploy(String deploymentConfigurationName) {.....}
def switchTraffic(String routeName, int greenWeight, int blueWeight) {....}
OpenShift Jenkins Pipeline (DSL) Plugin
Switch traffic to Green
node {
stage("build") {.....}
stage("deploy to test") {.....}
stage("manual testing") {.....}
stage("deploy blue on production") {.....}
stage("internal testing blue") {.....}
stage("swich traffic to blue") {.....}
stage("testing blue on production") {.....}
stage("deploy green on production") {.....}
stage("swich traffic to green") {
openshift.withCluster() {
openshift.withProject('app-prod') {
switchTraffic('app-green', 100, 0)
}
}
}
}
def deploy(String deploymentConfigurationName) {.....}
def switchTraffic(String routeName, int greenWeight, int blueWeight) {....}
OpenShift Jenkins Pipeline (DSL) Plugin
Let's run IT
#!/bin/bash
#Add permissions for jenkins
oc policy add-role-to-user edit system:serviceaccount:cicd:jenkins -n app
oc policy add-role-to-user edit system:serviceaccount:cicd:jenkins -n app-test
oc policy add-role-to-user edit system:serviceaccount:cicd:jenkins -n app-prod
Setup permissions for Jenkins
OpenShift Jenkins Pipeline (DSL) Plugin
Turn off the triggers
#!/bin/bash
#Turn off the triggers for dc in project app-test
oc set triggers dc/app-test --from-config --remove -n app-test
oc patch dc/app-test \
-p '[{"op": "replace", "path": "/spec/triggers/0/imageChangeParams/automatic", "value":false}]' \
--type=json -n app-test
#Turn off the triggers for dc in project app-prod
oc set triggers dc/app-green --from-config --remove -n app-prod
oc set triggers dc/app-blue --from-config --remove -n app-prod
oc patch dc/app-green -p ....
oc patch dc/app-blue -p ....
oc apply -f app-pipeline.yaml
Apply pipeline to OpenShift
OpenShift Jenkins Pipeline (DSL) Plugin
Zapraszam na
Warszawa 02.06.2020
CICI on OpenShift
By andipansa
CICI on OpenShift
- 199