Jenkins Overview

Resources

Introduction

Continuous Integration is the automation of building the code and running automate test against the build. This process may take from pushing code to a remote repository or a developer's Pull Request to the merging changes to the main branch.

What is CI from CI/CD?

Continuous Delivery is adding to the CI process an automation for getting ready a new release to deploy at any moment by clicking on a button.

Continuous Deployment is almost the same as Continuous Delivery. The no needed human intervention is the unique difference with Continuous Delivery. Once a new release pass the automated tests this will be automatically deployed to production.

What is CD from CI/CD?

What is Jenkins?

Jenkins is a self-contained, open source automation server which can be used to automate all sorts of tasks related to building, testing, and delivering or deploying software.

 

Jenkins' Documentation

Jenkins is a self-contained, open source automation server which plays the middleman role between a software project, organisation or company's toolset used to automate all sorts of tasks related to building, testing, and delivering or deploying software.

 

Being NOT a service allows to run it on-premise, in the cloud or wherever is needed.

What is Jenkins to me?

...and many more!

Jenkins Architecting

Technically speaking Jenkins is a Java-based Web Application. Just a .war file usually executed through the built in Jetty servlet container by:

This is the standalone configuration, being an only Jenkins Master, the scheduler and the builder.

java -jar jenkins.war

Jenkins supports the master-slave mode, where the building is performed by the slaves and the master will use its resources to only handle HTTP requests and manage the build environment.

 

Distributed builds

Slaves

  • Reduce Jenkins Master downtime on host maintenance actions
  • Avoid access to Jenkins Master sensitive information
  • Free up resources for Jenkins master

Advantages

What is a Jenkins Slave?

Technically speaking a Jenkins slave (also called agent or node) may be any device capable of running the agent.jar file.

The Jenkins slave will have as many:

  • Tools, resources and connectivity as the host where is running on has.
  • Permissions as the user used to launch the agent (something to be aware of)

JNLP

SSH

Communication Protocols

Demo Time

Scaling Jenkins

Achieving an optimal initial configuration for Jenkins Master is almost impossible since it is not an exact science. Even so there's a  few things to have in mind:

  • Jenkins does not require fast disk access to run well, but $JENKINS_HOME must be ready to scale fast.

 

  • Technically the memory range may be from 200 MB  to 70+ GB, but 4GB to 16GB is the most common range for a medium one.

Initial Configuration

The equation for estimating the maximum number of jobs, masters, and executors needed for an organization based on the number of developers is as follows:

number of jobs = number of developers * 3.333
number of masters = number of jobs / 500
number of executors = number of jobs * 0.03

These equations assume that the Jenkins master will have 5 cores with one core per 100 jobs (500 total jobs/master) and that teams will be divided into groups of 40.

Jenkins' documentation

version: '3'
services:
  master:
    image: jenkins/jenkins:2.138.2
    container_name: jenkins_master
    ports:
      - 80:8080
      - 50000:50000
    volumes:
      - jenkins-data:/var/jenkins_home
volumes:
  jenkins-data:

A Jenkins instance is scaled by adding more slave and scaling master vertically.

Scaling an Instance

Scaling master horizontally is actually not possible. Every Jenkins master instance is always a different instance.

Scaling Instances

Sometimes an only master can not face the load or having an only failure point is not viable for some teams because of the downtime risk.

When this happens every area, project or team may have its own Jenkins instance or even every environment.

Scaling Slaves

A Jenkins slave can be scaled vertically by adding more resources to the slave host. This will depend on the needs of the jobs to be run on.

It also can be scaled "horizontally" by adding more slaves. This will depend on the number of jobs to run.

Slaves Usage Optimization

A Jenkins Job will be able to do whatever the slave where is running on is able to do. Different pools of slaves can be created by using node labels based on:

  • Specific OS dependencies.
  • Tools only installed in some slaves.
  • Connectivity to some services located in private network

docker

macOs

windows

aws  linux

Tooling Slaves using Docker

Creating Slaves using Docker based on the jnlp-slave and ssh-slave official images is the best option. This approach brings:

  • Declarative Configuration
  • Changes Tracking
  • Easy Replacement
version: '3'
services:
  slave:
    image: jenkins/jnlp-slave:3.23-1
    container_name: jenkins_slave
    volumes:
      - slave-data:/home/jenkins/agent
    env_file:
      - slave.env
volumes:
  slave-data:
JENKINS_URL=
JENKINS_SECRET=
JENKINS_AGENT_NAME=
JENKINS_AGENT_WORKDIR=/home/jenkins/agent

Orchestrating Slaves

When the number of slaves increases exponentially maintaining and keeping them updated becomes a tedious work.

Swarm Plugin

Kubernetes Plugin

Demo Time

Jenkins Overview

By Marco Davalos

Made with Slides.com

Jenkins Overview

  • 511

More from Marco Davalos