Gradle in the Wild

Axel Springer - Best Practice Club

18 August 2015

Stoyan Stoyanov

welt.de

whoami

• Spreading the DevOps Cutlture @ Welt
• Weapons of choice
  • (Java), Ruby, Go
  • Chef, Gradle
• Twitter: @s_stoyanoff

Our Stack

• Legacy System
  • Escenic CMS
  • Java, Apache, Varnish, Tomcat, MySQL, Solr ...
  • Gradle, Jenkins, Chef, Vagrant
• Polyglot Micro Services
  • Java, Scala, Go
  • Gradle, Jenkins
  • Play 2, Spring Boot
  • AWS Beanstalk, Docker ..

The message

• Sharing experience
  • How we got to Gradle
  • How we're using the tool
• Gaining feedback on how you are using Gradle
• No maven bashing

Like any other software system, when build scripts are poorly designed and maintained, the effort to keep them working grows in what seems like an exponential manner.

 

(Continous Delivery Book - The Commit Stage)

The build process is important

• Testing is the most important part of CD
  • It's often underestimated
  • Integration of all kinds of test in the build should be easy
  • Fast feedback is fundamental
• Reproducable builds
• Release Engineer is an anti-pattern

The journey to Gradle

• Maven 3
• 30 Modules
  • 7000 LOC
  • A lot of Ant
• 15 min. for the build
• 50 min. for every release
• WTFs/h above the limit

Maven Release Plugin

Migrating

• gradlew init
  • Also creates the settings.gradle for us
• Scopes in Maven are identical to configurations in Gradle
• providedCompile only in the War-Plugin in Gradle
• Take all Ant-Targets as they are

configurations {
    provided
    testProvided.extendsFrom provided
}

sourceSets {
    main {
        compileClasspath += configurations.provided
    }

    test {
        compileClasspath += configurations.testProvided
    }
}

Comparing Artifacts

• Build with Maven first
• Build with Gradle
• Compare the artifacts with zipcmp (libzip)
• It feels like TDD
• Easy to automate in Gradle
• Helps to better understand the Maven build

apply plugin: 'war'

// some build logic here ..

war {
    includeEmptyDirs = false
}

war << {
    exec {
        ignoreExitValue = false
        executable 'zipcmp'
        args "$projectDir/target/$archiveName", archivePath
    }
}

Some Hints on Comparing

• Adapt maven-{jar,war}-plugin
  • <addMavenDescriptor>false</addMavenDescriptor>
• includeEmptyDirs=false in Gradle
• Manifests can also be adapted

Dependencies

• Break the build if any conflicts
  • failOnVersionConflict()
• ./gradlew dependencyInsight --dependency someDep
• Maven dependencyManagement can be handled with properties in Gradle

// versions.gradle
project.ext {
    escenicEngineVersion = '5.5.4.143753'
}

// dependencyDeclarations.gradle
apply from: "$rootDir/gradle/dependencies/versions.gradle"

project.ext.escenicNativeLibraries = [
    engineCore: "com.escenic.engine:engine-core:$escenicEngineVersion"
]

// in build.gradle
apply from: "$rootDir/gradle/dependencies/dependencyDeclarations.gradle"

// use the dependency in any subproject
compile escenicNativeLibraries.engineCore

Polyglot Project

• Ant-Targets from the Legacy-Build
  • Gradle -> Ant -> yuicompressor, less
  • Ant-Script very complex, slow, unmaintainable
  • JavaScript-Devs can't cope with Ant, Maven or Gradle

Integration of GruntJS

• Grunt ist JavaScript Task Runner (less, uglify, etc..)
• Plain JavaScript
• Creating Tasks on-the-fly (DRY)
• Dependency Management for JS with bower
• Very easy to integrate in Gradle

// in buildSrc
class Grunt extends DefaultTask {
    @Input
    List<String> commands

    @TaskAction
    void callGrunt() {
        project.exec {
            executable 'grunt'
            args commands
	}
    }
}

// in build.gradle
task installGruntDependencies(type: Exec) {
    inputs.file 'package.json'
    outputs.file 'node_modules'

    executable 'npm'
    args 'install'
}

task grunt(type: Grunt, dependsOn: installGruntDependencies) {
    commands = [project.name]
}

Tests in Gradle

• Adding Conventions for Integration Test is really easy
  • src/integTest/java/../*IntegTest.java
  • Run them only in Jenkins
• Unit-Tests are run
  • Only when the code, a test is covering, changes
  • Or when the test changes

Other Use Cases

• Assembling the Escenic-CMS
  • An upgrade costs a version bump and a click in Jenkins
  • Pushing a zip with all the provided dependencies into Nexus
• Mix use of local versioned dependencies in the project (from the zip) with others from mirrored Maven Central

Plugins

• Build-in plugins (java, war, maven-publish)
• idea, eclipse
• (gradle-tomcat-plugin)
• welt's gradle plugins

Lessons Learned

• Migrating an old legacy build, helps you identify a lot of problems
• Migrate first, optimize lately
  • Maven and Gradle at the same time is hard
  • Maven Know-How needed
• Don't migrate Ant-Scripts in Step 1

Lessons Learned (c'd)

• IDE integration not working out of the box
• A lot of Gradle plugins are crappy
• Beware the Gradle phases !
  • initialization vs configuration vs execution
• Task up-to-date problem
  • Always think about the inputs/outputs of a task

Lessons Learned (c'd)

• Build without clean works and is damn fast
  • Ok, only one clean per day in Jenkins ;)
• always use gradlew
• Configure HTTPS_PROXY in Jenkins globally
  • It helps for npm also
  • Use -Dhttps.proxyHost=foo -Dhttps.proxyPort=bar build

Conclusion

• Fast, reproducible, incremental builds
• No XML :)
• Easy to extend and support every possible scenario
• Best tool for the job

Next steps

• Expanding Know-How in teams
• Rethink of our current testing process
  • Less stages (stages on-demand)
  • Integration of functional tests
  • Manual tests should -> 0
  • Stay tuned for the next meetup :)

Must Reads

• Gradle in Action
• Gradle ein kompakter Einstieg

welt.de is hiring :)