Continuous Integration: Zero to Infinity

Pete Freitag

My Company:

Builds security products & services for CFML developers.

Conference Sponsor!

Agenda

What problems are we trying to solve?
What is Continuous Integration?
What do you need, and how do you set it up?
Show me some CFML specific things you can do.

Does this sound like you...

One beautiful spring afternoon

You made a simple code fix

So simple, it was only one line

In fact it was so simple, you just deployed to production without testing it

Did I mention it was Friday?

Just as you were about to eat dinner

Can you guess what happens next?

You broke it, and now you have to fix it

Let's see how CI might have prevented this one

What problems does CI solve?

Find Problems Early
- Less time fixing problems
- Ship Higher Quality
Automates things you don't want to do
- Run your tests
- Check code / dependencies for security issues

Setting up CI is an investment

Without CI

With CI

If it hurts do it often

and it will hurt less

More Smaller Changes

You can deliver smaller more frequent releases
- Easier to fix when less is changed
- Frequent wins for developers
- Hard / Impossible to do without CI
- Learn from users faster

What do you need?

Source Code
A CI Server

Version Control

If you are not yet using version control yet, head over to github.com or gitlab.com and create a free account.
- "version control is not a fad" -- Pete Freitag
Technically you can do CI without version control, but it really works best with version control.
You don't need to use git, though most everything you read will assume you are using git.

What does the CI Server Do?

There are two basic functions.

What does a CI Server Do?

Waits for an event to occur (a trigger)
- A Trigger Might be:
  - Every time you commit code
  - On Pull Request
  - Scheduled: Nightly, weekly, etc.

What does a CI Server Do?

Executes code (a task) when the trigger event occurs.
- The task can pass or fail
- It can depend on success of prior tasks
- The task can create output, or artifacts
- The task can deploy your code to another server
- The CI servers provides a clean execution environment with your latest source code.
- The task can do whatever you want, it is just code!

Selecting a CI Server

CI / CD Server Options

Github Actions
Bitbucket Pipelines
GitLab CI / CD
AWS CodeBuild / CodePipeline
Azure DevOps
CircleCI
TravisCI
Buddy

TeamCity
Go CD
Bamboo
Spinnaker
CodeShip
Wrecker
Semaphore
And more...

Dozens of CI Servers

How do you pick?
- Does my version control service provide it?
  - ✅ GitHub
  - ✅ GitLab
  - ✅ BitBucket

Picking a CI Platform

Must it run onsite? or can someone else host it?
- Pick cloud hosted unless you have to run onsite
Do I have resources to keep onsite software running?
- Needs periodic security patching, updating
Cost
- Probably cheaper let someone else host it

Picking a CI Platform

What OS's are supported?
- Windows, Mobile Phones, etc
Does my cloud provider have a CI service?
- AWS: CodeBuild, CodePipeline
- Azure: Azure DevOps
- Might be handy for CD (Continuous Deployment)
Ease of Use vs Popularity
- Jenkins hard to use but popular
- Buddy easy to use but not as popular

Nice to have features

Secrets Store
- Lets you store API keys, ssh keys for deploying, etc.
UI for Test Results
- GitLab Example: https://github.com/foundeo/fixinator/wiki/Running-Fixinator-on-GitLab
- BitBucket Example: https://bitbucket.org/pfreitag/fixinator-example-fail/
Matrix Builds

What do you need to know?

YAML
- Know that tabs are important
Shell Scripting
- Bash, Powershell, Bat files
Or just know how to Copy / Paste
- You can copy everything I show you today

What else do you need to know?

First Hand Experience
- You need to try it yourself
- Only then will you fully understand it
It will fail, be patient or ask for help
- See: https://github.com/foundeo/cfml-ci-examples/commits/master

Our CI Example App

https://github.com/foundeo/cfml-ci-examples/

The mission critical code

component {

    remote function add(x, y) {
        return x+y;
    }

}

services/calc.cfc

It would be great to

Automatically tell me if:
- Any CFML compiler errors exist
- Any security vulnerabilities exist
- I broke the primary function of the site

But First, Hello World

name: HelloCI

# The trigger
on:
  push:
    branches: [ hello ]

# The executable
jobs:
  build:
    runs-on: ubuntu-latest
    steps:
    # Runs a single command using the runners shell
    - name: Run a one-line script
      run: echo Hello, world!

https://github.com/foundeo/cfml-ci-examples/blob/hello/.github/workflows/hello.yml

Next we will add:

CFML Compiler	Fixinator	Postman/Newman
curl	puppeteer	cflint
textlint	codechecker	TestBox

Keep in Mind

Each tool adds a different value
- You don't need to use them all
Some tools will be easy to add
Some will add value right away
- Some require you to do some work first (eg write tests)

CFML Compiler

Runs the Lucee compiler on all your code
- Exact lucee version
Find compiler errors in the code such as:
- Missing a closing tag (cfoutput, cfif, etc)
- Too many / too few parenthesis
Minimal effort, lots of benefits

https://github.com/foundeo/cfml-compiler

Compiler Github Action

jobs:
  cfcompile:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v2
    - name: cfcompile
      uses: foundeo/cfml-compiler-action@master
      with:
        cfengine: lucee@5.2.9.31

https://github.com/foundeo/cfml-compiler-action

Fixinator

Scans CFML Code for Security Issues
- SQL Injection, Remote Code Execution, File Path issues, vulnerable dependencies, etc.
Auto fix vulnerabilities

https://fixinator.app

Easy to add

You don't have to write any code / tests
Guides for all popular CI platforms

https://github.com/foundeo/fixinator/wiki/Continuous-Integration-Guide

Fixinator Github Action

jobs:
  fixinator:
    runs-on: ubuntu-latest
    steps: 
    - uses: actions/checkout@v2
    - name: Fixinator Security Scan 
      uses: foundeo/fixinator-github-action@master
      with:
        fixinator_api_key: ${{ secrets.FIXINATOR_API_KEY }}

https://github.com/foundeo/fixinator-github-action/

Curl + CommandBox

Use CommandBox to start a server
- box server start
Make a request to your server using curl
- Check status code, or result
Some what easy to do

Postman / Newman

Postman a GUI Tool for Writing HTTP Tests
- Really good for REST APIs
Newman is the CLI version of postman
- Export your Postman tests and run on Newman

Puppeteer

Provides an API to control a Browser from JavaScript
- Fill in forms fields
- Click buttons
- Get the value of a div
- Take a screenshot
- Anything you can imagine

Example Puppeteer Script

const puppeteer = require('puppeteer');

(async() => {

    const browser = await puppeteer.launch();

    const page = await browser.newPage();
    await page.goto('http://127.0.0.1:8855/', {waitUntil: 'load'});

    await page.type('#x', '5');
    await page.type('#y', '3');

    await page.click('#addButton');

    setTimeout(async()=> {
        const answer = await page.$eval('#z',  el => { return el.value } );

        await page.screenshot({path: 'screenshot.png'});
        if (answer == "8") {
            console.log("Value Was 8");
            await browser.close();
            process.exit(0);
        } else {
            console.log("Value was not 8, it was: " + answer);
            await browser.close();
            process.exit(1);
        }
        
    }, 200);
    
})();

Puppeteer

Recorders
- If you don't want to write a script you can install a browser extension to record a script.
Example Scripts
- https://github.com/checkly/puppeteer-examples

TestBox

Write Tests in CFML
- BDD (Behavior Driven Development)
- xUNIT
Built by Ortus Solutions

Example BDD Test

component extends="testbox.system.BaseSpec" {

  function run(testResults, testBox) {
		
    describe("Test Addition", function() {
      it("Should be able to add two positive integers", function() {
        var calc = new services.calc();
        expect(calc.add(5,3)).toBe(8);
      });

    });
        
  }

}

TestBox CheatSheet: https://cfdocs.org/testbox

CodeChecker & CFLint

Code Scanners that look for:
- Code Quality Issues
- Code Style Issues (eg require semicolons)
- Performance Issues
- and more
Configurable to Include / Exclude Rules

Textlint

Example of a non CFML tool
It turned out to be not super CI friendly
- Probably better for running locally
Lots of configurable rules
- But you have to install each one via npm
- Does not come with any rules by default

Just scratching the surface

CFML Compiler	Fixinator	Postman/Newman
curl	puppeteer	cflint
textlint	codechecker	TestBox

Dig Deeper Today

Do It with CFML: Building Your Own Tools with CommandBox
- Matthew Clemente, Today @ 1pm CT
Just Do It! ColdBox Integration Testing
- Javier Quintero, Today @ 3:10pm CT
5 CommandBox Modules you should be using
- Brad Wood, Today @ 4:10pm CT

Dig Deeper Tomorrow

End To End Test Automation using TestCafè
- Francisco Mancardi, Tomorrow @ 1pm CT
Testing My Non-ColdBox Site With TestBox
- Nolan Erck, Tomorrow @ 4:10pm CT

Thank You!

Questions?

foundeo.com | fuseguard.com | fixinator.app | hackmycf.com

Continuous Integration: Zero to Infinity

Pete Freitag

Agenda

Does this sound like you...

One beautiful spring afternoon

You made a simple code fix

So simple, it was only one line

In fact it was so simple, you just deployed to production without testing it

Did I mention it was Friday?

Just as you were about to eat dinner

Can you guess what happens next?

You broke it, and now you have to fix it

Let's see how CI might have prevented this one

What problems does CI solve?

Setting up CI is an investment

If it hurts do it often

and it will hurt less

More Smaller Changes

What do you need?

Version Control

What does the CI Server Do?

What does a CI Server Do?

What does a CI Server Do?

Selecting a CI Server

CI / CD Server Options

Dozens of CI Servers

Picking a CI Platform

Picking a CI Platform

Nice to have features

What do you need to know?

What else do you need to know?

Our CI Example App

The mission critical code

It would be great to

But First, Hello World

Next we will add:

Keep in Mind

CFML Compiler

Compiler Github Action

Fixinator

Easy to add

Fixinator Github Action

Curl + CommandBox

Postman / Newman

Puppeteer

Example Puppeteer Script

Puppeteer

TestBox

Example BDD Test

CodeChecker & CFLint

Textlint

Just scratching the surface

Dig Deeper Today

Dig Deeper Tomorrow

Thank You!

Continuous Integration: Zero to Infinity

More from Pete Freitag