Integration Testing with

Docker and Testcontainers

 

  • Head of Blockchain Lab @ Institute for Internet Security
    • PhD Student @ RWTH Aachen
  • Consultant @ Styrascosoft
  • Testcontainers Maintainer and Open Source Enthusiast
  • Oracle Groundbreaker Ambassador
  • Organizer Software Craftsmanship Meetup Ruhr

Kevin Wittek          @kiview

Why do we Test?

Key learnings

  • Easy setup of complete test environments
  • Not a lot effort to integration test using real external applications
  • Avoid integrated tests!
$ git clone https://github.com/foo/bar.git
$ cd bar
$ ./gradlew build

Source: https://watirmelon.blog/2012/01/31/introducing-the-software-testing-ice-cream-cone/

My definiton

For example:

  • Launch application server
  • Framework/Library interactions
  • Interact with external ports (e.g. network, file system, database) 

"Tests which interact with external systems/dependencies"

But why a pyramid?

Source: https://labs.spotify.com/2018/01/11/testing-of-microservices/

Testing Honeycomb

Source: https://labs.spotify.com/2018/01/11/testing-of-microservices/, http://blog.thecodewhisperer.com/permalink/integrated-tests-are-a-scam

Integrated Tests

"I use the term integrated test to mean any test whose result (pass or fail) depends on the correctness of the implementation of more than one piece of non-trivial behavior." -  J.B. Rainsberger

"A test that will pass or fail based on the correctness of another system." - Spotify

Source: https://labs.spotify.com/2018/01/11/testing-of-microservices/, http://blog.thecodewhisperer.com/permalink/integrated-tests-are-a-scam

Integrated Tests

For example:

  • We (manually) spin up other services in a local testing environment
  • We test against other services in a shared testing environment
  • Changes to your system breaks tests for other systems

Which tools to use?

(in the JVM world and beyond)

Source: https://www.docker.com/what-container

Source: https://www.docker.com/what-container

Source: https://www.docker.com/what-container

Why combine?

  • Easy setup of dev environment
  • Uniform build and test environments
    • Also self contained and portable!
  • No installation and setup of external software
    • Except Docker 😜

How to fuse both worlds?

GenericContainer redis =
    new GenericContainer("redis:3.0.2")
               .withExposedPorts(6379);

redis.start();

// test my stuff

redis.stop();

The Project

  • testcontainers-java first released in 2015
  • 100% OSS, MIT licensed
  • 65 releases, 149 contributors
  • Core maintainers
    • Richard North
    • Sergei Egorov
    • Kevin Wittek
  • Forks in Python, C#, Rust, Go, JS; Scala wrapper (+ unofficial Clojure wrapper 😁)

@whichrich

Capabilities

  • Generic docker container support
    • use any Docker image to support tests
  • Databases (many!) and Stream processing (Kafka, Pulsar)
  • AWS mocks (Localstack)
  • Docker Compose
    • Or Docker Compose alternative!
  • Selenium
  • Chaos testing (toxiproxy)

Capabilities (2)

  • Dynamic port binding and API
  • WaitStrategies
  • Docker environment discovery (e.g. docker-machine, DOCKER_HOST, Docker for Mac, Docker for Windows)
  • Platform independent
    • Linux, macOS, Windows 10 (with NPIPE support!)

@ashleymcnamara

Works on Windows too!

JDK14 Compatible!
+ regularly build against openjdk-ea

Support all Major CI Systems!

  • Jenkins
  • TravisCI
  • GitlabCI
  • CircleCI
  • Azure DevOps Pipelines
  • Bitbucket
  • GitHub Actions
  • ...
// Set up a redis container
@ClassRule
public static GenericContainer redis =
    new GenericContainer("redis:3.0.2")
               .withExposedPorts(6379);

JUnit4 Rules

Spock Extension


@Testcontainers
class TestContainersClassIT extends Specification {


    @Shared
    GenericContainer genericContainer = 
       new GenericContainer("postgres:latest")
            .withExposedPorts(5432)
            .withEnv([
                POSTGRES_USER: "foo"
                POSTGRES_PASSWORD: "secret"
            ])
}

Testcontainers-Jupiter (JUnit5)

@Testcontainers
class SomeTest {

    @Container
    private MySQLContainer mySQLContainer = new MySQLContainer();

    @Test
    void someTestMethod() {
        String url = mySQLContainer.getJdbcUrl();
         // create a connection and run test as normal
    }
}

Singleton Container Pattern

abstract class AbstractContainerBaseTest {

    static final MySQLContainer MY_SQL_CONTAINER;

    static {
        MY_SQL_CONTAINER = new MySQLContainer();
        MY_SQL_CONTAINER.start();
    }
}

class FirstTest extends AbstractContainerBaseTest {

    @Test
    void someTestMethod() {
        String url = MY_SQL_CONTAINER.getJdbcUrl();

        // create a connection and run test as normal
    }
}

Testcontainers 1.14.0 Released!

Auto generated GHA

Auto generated GHA

Test Matrix

task testMatrix {
    project.afterEvaluate {
        def checkTasks = subprojects.collect {
            it.tasks.findByName("check")
        }.findAll { it != null }

        dependsOn(checkTasks)
        doLast {
            // Obtain a list of check tasks that are not up-to-date, i.e.
            //  the ones which Gradle cannot find a cached output for.
            def checkTaskPaths = checkTasks
                .findAll { !it.state.upToDate }
                .collect { it.path }

            println(JsonOutput.toJson(checkTaskPaths))
        }
    }
}

Spring 5.2.5

@DynamicPropertySource

@SpringBootTest
@Testcontainers
class ExampleIntegrationTests {

    @Container
    static Neo4jContainer<?> neo4j = new Neo4jContainer<>();

    @DynamicPropertySource
    static void neo4jProperties(DynamicPropertyRegistry registry) {
        registry.add("spring.data.neo4j.uri", neo4j::getBoltUrl);
    }

}

Docker-Compose


version: "3"

services:
  vote:
    image: vote-frontend
    command: python app.py
    volumes:
     - ./vote:/app
    ports:
      - "5000:80"

  redis:
    image: redis:alpine
    ports: ["6379"]

  worker:
    image: worker

  db:
    image: postgres:9.4

  result:
    image: result-frontend
    command: nodemon --debug server.js
    volumes:
      - ./result:/app
    ports:
      - "5001:80"
      - "5858:5858"

TC as a DockerCompose replacement

https://bsideup.github.io/posts/local_development_with_testcontainers/

CI inside containers?

Docker in Docker - DinD

Docker Wormhole

Docker Wormhole

Future

and Common Pitfalls...

SELF-typing is sometimes a PITA

  • Pattern defined in "Effective Java"
    • Generic type with a recursive type paramter
    • MyClass<T extends MyClass<T>>
  • Doesn't play nice with other JVM languages (Scala, Kotlin)
  • Confusing for new users and contributors
  • Not consistent with all modules

Ideas

  • Using functional Builder-Pattern
    • Maybe Lombok @SuperBuilder?
  • Double Brace Initialization
    • People don't like this 😱

Double Brace Initialization 😜

private GenericContainer myContainer = new GenericContainer("myImage:42.23") {{
   withExposedPorts(4711);

   if (System.getenv().get("INSIDE_CI") == null) {
       org.testcontainers.Testcontainers.exposeHostPorts(8080);
   }

}};

TDD workflows

  • Container startup introduces overhead
  • Testcontainers should be ephemeral
  • We still want to avoid test pollution and integrated tests

Moar Future

  • Testcontainers 2.0
  • Complete Podman support
    • Working with RedHat on making the TC test suite feature parity reference test suite :)
  • Container-Core
    • Test-Framework agnostic
    • OO abstraction of Docker containers
  • Other languages
    • More languages and aligning existing forks
    • GraalVM (mind=blown)

Far Future...

TODO: add Safe Harbor Statement here

  • Windows Containers on Windows (WCOW)
  • Orchestrators (Docker Swarm, Kubernetes)
    • Maybe one day... (Help from experts welcome!)
  • Other container engines (e.g. Podman, Firecracker)
  • Build and test inside containers with full IDE support
    • Cloud IDEs (Codespaces?)
    • Visual Studio Code Remote Development
    • Support for lightweight dev workstations

GitHub Sponsors

Made with Slides.com