Test Jenkins Pipeline And SHared LIBRARIES

Stanislav Ovchar

stchar

pipeline

Jenkins

  • Simple pipeline
  • in-browser editor 
  • Manual testing

pipeline script

JobDSL/

JenkinsFile

Jenkins

shared libs

  • scm
  • Automated deploy
  • Manual testing

problemS

  • more jobs = more code = more efforts
     
  • Jenkins out of the box has a lack of framework or tools to debug and test
     
  • Integration with Jenkins plugins should be verified

E2E Tests

Unit tests 

Integration
Tests

Sandbox project

 

Split big pipeline in small pieces

 

load'em, test'em one by one

 

E2E Tests

Unit tests 

Integration
Tests

@JenkinsRule

@WithPlugin

@WithTimeout

 

Verify how your pipeline integrates with jenkins plugins

E2E Tests

Unit tests 

Integration
Tests

Mock (sh, git, parallel, shared lib)

 

Check pipeline script logic

Jenkins PIpeline UNIT

2017 https://www.youtube.com/watch?v=RmrpUtbVR7o

https://github.com/jenkinsci/JenkinsPipelineUnit

ozangunalp

  • jUnit, Spock?
  • Load & run pipeline scripts
  • Mock pipeline methods
  • Load & run shared library classes
  • Trace Call Stack
  • CPS

CAN DO

  • Declarative Pipeline
  • Load & run pipeline plugins
  • Load scripts into Jenkins Test Harness 
  • Static analysis
  • Coverage

CAN NOT DO

Jenkins SPOCK

2018 https://www.youtube.com/watch?v=4PZ-UFBexIE

 https://github.com/homeaway/jenkins-spock 

awittha

  • Support Spock
  • Load & run pipeline scripts
  • Static analysis
  • Mock pipeline methods
  • Load & run shared library classes
  • Trace Call Stack

CAN DO

  • Load & run pipeline plugins
  • Load scripts into Jenkins Test Harness
  • Declarative Pipeline ?
  • Coverage ?
  • CPS ?

CAN NOT DO

JENKINS PIPELINE UNIT

script

Test

BasePipelineTest

helper

binding

# build.gradle
# compile 'junit:junit:4.12'
# compile 'org.assertj:assertj-core:3.4.1'
# compile 'com.lesfurets:jenkins-pipeline-unit:1.1'

#> gradle test

script

Test

BasePipelineTest

helper

binding

def greet = "Hello World!"
node() {
  echo greet 
}

script

Test

BasePipelineTest

helper

binding

registerAllowedMethod('readFile',[Map.class],{
 // load local file from resources
})
runScript('script')

script

Test

BasePipelineTest

helper

binding

setVariable('env',[BUILD_URL:'http://example.com'])
getVariable('some_var')

script

Test

BasePipelineTest

helper

binding

void setUp() {
  ...
  helper.registerAllowedMethod("stage", 
    [String.class, Closure.class], null)
  
  helper.registerAllowedMethod("write", 
    [String.class, Closure.class], null)
  ...
}        

script

Test

BasePipelineTest

helper

binding

@Test
void testHello() {
    def script = runScript("hello.groovy")
    assertJobStatusSuccess()
    printCallStack()
}
// printCallStack()

       hello.run()
          hello.node(groovy.lang.Closure)
             hello.echo(Hello World!)
class PipelineTestHelper {
  protected Map<MethodSignature, Closure> allowedMethodCallbacks

  Script loadScript(String scriptName, Binding binding) {
    ...
    Script script = InvokerHelper.createScript(scriptClass, binding)
    script.metaClass.invokeMethod = getMethodInterceptor()
    script.metaClass.static.invokeMethod = getMethodInterceptor()
    script.metaClass.methodMissing = getMethodMissingInterceptor()
    return script
  }

  registerAllowedMethod(...) {
    allowedMethodCallbacks.put(...)
  }
}

Integration tests

Jenkins test HARNESs

https://github.com/jenkinsci/jenkins-test-harness

https://wiki.jenkins.io/display/JENKINS/Unit+Test

https://javadoc.jenkins.io/component/jenkins-test-harness/

 

 

 

  • Support JenkinsPipelineUnit
  • Resolves plugin dependencies for JTH

gradle plugin

script

Test

gradle-plugin

plugins

JTH

def greet = "Hello World!"
node() {
  input message: 'continue?', id: 123
  echo greet
  emailext(to: john.doe@example.com, body: greet) 
}

script

Test

gradle-plugin

plugins

JTH

sharedLibrary {
   pluginDependencies {
     dependency("org.jenkins-ci.plugins", 
       "pipeline-input-step", "2.8")
     
     dependency("org.jenkins-ci.plugins", 
       "email-ext", "2.63")
   }
 }

script

Test

gradle-plugin

plugins

JTH

class TestReleaseMail {
   ...
   @Rule
   public JenkinsRule rule = new JenkinsRule() {
   ...

}

script

Test

gradle-plugin

plugins

JTH

class TestReleaseMail {
   @Test
   void "verify release mail"() {
     
     WorkflowJob workflowJob = rule.createProject(WorkflowJob, 
        'test-release-mail')
     def script = new File('jobs/release/mail/basic.groovy')
     
     workflowJob.definition = 
       new CpsFlowDefinition(script.text, true)
     ...
   }
   ...
}

script

Test

gradle-plugin

plugins

JTH

class TestReleaseMail {
   @Test
   void "verify release mail"() {
     ...
     WorkflowRun result = rule.buildAndAssertSuccess(workflowJob)
     
     rule.assertLogContains('Sending email to: jhon.doe@example.com', 
       result)
 
     assertEquals(1, 
      Mailbox.get("jhon.doe@example.com").getNewMessageCount());
     ...
   }
   ...
}

TeSt Shared Libraries

pipeline-template

Unit-Tests

  • template to load the lib 

 

 

  • cp -r src vars  miylib@master
  • gradle check

 

  • test/unit/groovy
  • JenkinsPipelineUnit

 

  • test/integration/groovy
  • @JenkinsRule + gradle-plugin

src, vars

Integration

Tests

Demo Shared Lib Test (gradle)

stchar/pipeline-sharedlib-testharness

#> gradle check
#> git clone https://github.com/stchar/pipeline-sharedlib-testharness.git


.
├── jobs                              # Template pipeline scripts to load the lib
│   │                                 # are used by unit tests
│   └── template
|       |
│       └── template.groovy
├── src
│   └── org
│       └── hcm
│           └── libjenkins
│               └── *.groovy         # Examples of Library class
├── test
│   ├── integration
│   │   └── groovy
│   │       └── *.groovy             # Integration tests
│   └── unit
│       └── groovy
│           └── *.groovy             # JenkinsPipelineUnit tests
└── vars
    └── *.groovy                     # Jenkins pipeline shared library vars objects
// file: jobs/template/gitlab.groovy

@Library('jenkins-commons')
import org.hcm.libjenkins.Gitlab

gitlab_lib = new Gitlab(this)

gl_stages = ['hello']

gitlabBuild(gl_stages) {
  gitlab_stage(gl_stages,'hello','master',10) {
    echo "Hello World!"
  }
}

return this
// file: test/unit/groovy/TestJenkinsCommonLib.groovy

...

class TestJenkinsCommonLib extends BasePipelineTest {

  String sharedLibs = ''

  @Override
  @Before
  void setUp() throws Exception {
      scriptRoots += 'jobs'
      super.setUp()
      def library = library().name('jenkins-commons')
                       .defaultVersion("master")
                       .allowOverride(true)
                       .implicit(true)
                       .targetPath('build/libs')
                       .retriever(localSource('build/libs'))
                       .build()
      helper.registerSharedLibrary(library)
  }

 
}
// file:  test/unit/groovy/TestJenkinsCommonLib.groovy

 ...

  @Test
  void should_execute_without_errors() throws Exception {
    def script = runScript("template/pipeline/template.groovy")
    //printCallStack()
  }

  @Test
  void verify_is_upstream() throws Exception {
    def script = runScript("template/pipeline/template.groovy")
    assertEquals('Verify is_upstream ', false, script.gitlab_lib.is_upstream('feature'))
    assertEquals('Verify is_upstream ', true, script.gitlab_lib.is_upstream('master'))
    assertEquals('Verify is_upstream ', true, script.gitlab_lib.is_upstream('rel-1.2.3'))
    //printCallStack()
  }
// file:  test/integration/groovy/TestReleaseMail.groovy
 
class TestReleaseMail {  
  @Rule
  public JenkinsRule rule = new JenkinsRule() {
    @Override
    public void before() throws Throwable {
      super.before()
      Mailbox.clearAll()
    }

    @Override
    public void after() throws Exception {
      super.after()
      Mailbox.clearAll()
    }
  }
...
}
// file:  test/integration/groovy/TestReleaseMail.groovy
 
class TestReleaseMail {  
  ...
  @Test
  void "verify release mail"() {

    WorkflowJob workflowJob = rule.createProject(WorkflowJob, 'test-release-mail')
    def script = new File('jobs/release/mail/basic.groovy')
    workflowJob.definition = new CpsFlowDefinition(script.text, true)

    WorkflowRun result = rule.buildAndAssertSuccess(workflowJob)
    
    rule.assertLogContains('Sending email to: jhon.doe@example.com', result)

    assertEquals(1, Mailbox.get("jhon.doe@example.com").getNewMessageCount());
    
    Message message = Mailbox.get("jhon.doe@example.com").get(0);
    MimeMultipart part = (MimeMultipart) message.getContent();
    
    assertTrue("Verify message body", 
       part.getBodyPart(0).getInputStream().text.contains("Release Artifacts"))


  }
  ...
}

LINKS

  • https://github.com/lesfurets/JenkinsPipelineUnit
  • https://github.com/mkobit/jenkins-pipeline-shared-libraries-gradle-plugin
  • https://github.com/jenkinsci/jenkins-test-harnes

  • https://github.com/stchar/pipeline-sharedlib-testharness
Made with Slides.com