Testcontainers

A year (or more) in review

  • ​Blockchain Research Group Lead @ Institute for Internet Security
    • PhD Student @ RWTH Aachen
  • Consultant & Coach
  • Testcontainers Maintainer and Open Source Enthusiast
  • Blockchain Activities
    • ​Bloxberg Consortium
    • Sovrin Steward Council
  • Oracle Groundbreaker Ambassador
  • Used to be model for Opel Manta Calendar

Kevin Wittek          @kiview

Thanks to Sergei!

(for slides, idea and everything else)

@bsideup

Why do we Test?

Ariane 5 (Report on Inquiry Board)

The internal SRI* software exception was caused during execution of a data conversion from 64-bit floating point to 16-bit signed integer value. The floating point number which was converted had a value greater than what could be represented by a 16-bit signed integer.

 

Ariane 5 (Recommendations)

R10 Include trajectory data in specifications and test requirements.

R11 Review the test coverage of existing equipment and extend it where it is deemed necessary.

R12 Give the justification documents the same attention as code. Improve the technique for keeping code and its justifications consistent.

R13 Set up a team that will prepare the procedure for qualifying software, propose stringent rules for confirming such qualification, and ascertain that specification, verification and testing of software are of a consistently high quality in the Ariane 5 programme. Including external RAMS experts is to be considered.

How I ended up on a dairy farm to patch a JEE application...

Source: https://watirmelon.blog/2012/01/31/introducing-the-software-testing-ice-cream-cone/

But why a pyramid?

Source: https://labs.spotify.com/2018/01/11/testing-of-microservices/

Testing Honeycomb

Source: https://labs.spotify.com/2018/01/11/testing-of-microservices/, http://blog.thecodewhisperer.com/permalink/integrated-tests-are-a-scam

Integrated Tests

"I use the term integrated test to mean any test whose result (pass or fail) depends on the correctness of the implementation of more than one piece of non-trivial behavior." -  J.B. Rainsberger

"A test that will pass or fail based on the correctness of another system." - Spotify

Source: https://labs.spotify.com/2018/01/11/testing-of-microservices/, http://blog.thecodewhisperer.com/permalink/integrated-tests-are-a-scam

Integrated Tests

For example:

  • We (manually) spin up other services in a local testing environment
  • We test against other services in a shared testing environment
  • Changes to your system breaks tests for other systems
GenericContainer redis =
    new GenericContainer("redis:3.0.2")
               .withExposedPorts(6379);

redis.start();

// test my stuff

redis.stop();

The Project

  • testcontainers-java first released in 2015
  • 100% OSS, MIT licensed
  • 65 releases, 149 contributors
  • Core maintainers
    • Richard North
    • Sergei Egorov
    • Kevin Wittek
  • Forks in Python, C#, Rust, Go, JS; Scala wrapper

@whichrich

Capabilities

  • Generic docker container support
    • use any Docker image to support tests
  • Databases (many!) and Stream processing (Kafka, Pulsar)
  • AWS mocks (Localstack)
  • Docker Compose
  • Selenium
  • Chaos testing

Capabilities (2)

  • Dynamic port binding and API
  • WaitStrategies
  • Docker environment discovery (e.g. docker-machine, DOCKER_HOST, Docker for Mac, Docker for Windows)
  • Platform independent
    • Linux, macOS, Windows 10 (with NPIPE support!)

@ashleymcnamara

Works on Windows too!

JDK15 (?) Compatible!
+ regularly build against openjdk-ea

Users

Spock Extension


@Testcontainers
class TestContainersClassIT extends Specification {


    @Shared
    GenericContainer genericContainer = 
       new GenericContainer("postgres:latest")
            .withExposedPorts(5432)
            .withEnv([
                POSTGRES_USER: "foo"
                POSTGRES_PASSWORD: "secret"
            ])
}
// Set up a redis container
@ClassRule
public static GenericContainer redis =
    new GenericContainer("redis:3.0.2")
               .withExposedPorts(6379);

JUnit4 Rules

Testcontainers-Jupiter (JUnit5)

@Testcontainers
class SomeTest {

    @Container
    private MySQLContainer mySQLContainer = new MySQLContainer();

    @Test
    void someTestMethod() {
        String url = mySQLContainer.getJdbcUrl();
         // create a connection and run test as normal
    }
}

What happened so far...

Sep, 2018 (1.9.x)

  • OkHttp by default
  • Windows npipe support
  • Registry auth support on Windows
  • Fix local compose on Windows
  • Host ports exposing
Testcontainers.exposeHostPorts(localServerPort);
final String rootUrl =
    String.format("http://host.testcontainers.internal:%d/", localServerPort);

Sep, 2018 (1.9.x)

  • Dynamic port binding in Couchbase module
  • New modules
    • ClickHouse
    • PostGIS

Nov, 2018 (1.10.x)

  • JUnit5 (Jupiter) support
  • Documentation relaunch
    • Code examples from as part of CI
  • ENV var to turn off Ryuk (TESTCONTAINERS_RYUK_DISABLED=true)
  • shm + tmpfs settings
  • Dependabot
  • New modules
    • Neo4j & Elasticsearch
You can see an example test that could 
have been written for it (without using Testcontainers):

<!--codeinclude-->
[Pre-Testcontainers test code]
(../examples/junit5/redis/src/test/java/quickstart/RedisBackedCacheIntTestStep0.java) 
block:RedisBackedCacheIntTestStep0
<!--/codeinclude-->

Dependabot PRs

Dependabot FTW!

Mar, 2019 (1.11.x)

  • Chaos testing support (toxiproxy)
  • fsync=off for PostgreSQL module
  • Drop Netty transport
  • Reworking shading

Jul, 2019 (1.12.x)

  • dependsOn API
  • Improved pull handling
  • CI for Windows!
    • Azure Pipelines with private Windows node
  • New modules
    • DB2, CockroachDB & RabbitMQ
GenericContainer container = new GenericContainer()
        .dependsOn(postgresContainer, kafkaContainer);

Jul, 2019 (1.12.x)

  • Secure logging defaults for properties and environment variables
  • Use official Oracle JDBC driver
  • Support for DockerClient event streaming
  • Docker Compose --build support
  • Image pull policies (AgeBasedPullPolicy) 
new GenericContainer<>(imageName)
        .withImagePullPolicy(PullPolicy.alwaysPull())

AgeBasedPullPolicy oneHour = new AgeBasedPullPolicy(
  Duration.of(1L, ChronoUnit.HOURS)
);
new GenericContainer<>(imageName)
        .withImagePullPolicy(oneHour)

Mar, 2020 (1.13.x)

  • Access Docker Compose containers by service name
  • New modules
    • Presto & OrientDB
public DockerComposeContainer environment = 
  new DockerComposeContainer(new File("src/test/resources/compose-test.yml"))
        .withExposedService("redis_1", REDIS_PORT)
        .withExposedService("db_1", 3306);

Optional<ContainerState> result = environment.getContainerByServiceName("db_1");

Apr, 2020 (1.14.x)

  • docker-java updated to 3.2.x (deprecations!)
  • R2DBC support
  • Improved Couchbase support
  • JUnit Jupiter and Spock is now TestLifecycleAware
    • Supporting VNC recording
r2dbc:tc:mysql:///databasename?TC_IMAGE_TAG=5.7.22

Apr, 2020 (1.14.x)

Jul, 2020 (1.15.x RC)

  • Support for rootless Docker!
  • Deprecation of ambiguous constructors
  • Un-shade docker-java-api
  • Optional Apache HttpClinet 5 transport

// deprecrate
new XyzContainer()
new XyzContainer(String)

// in favor for
new XyzContainer(DockerImageName)

// allowing
new XyzContainer( DockerImageName.parse( "the/image:tag" ) )

Future

and Common Pitfalls...

SELF-typing is sometimes a PITA

  • Pattern defined in "Effective Java"
    • Generic type with a recursive type paramter
    • MyClass<T extends MyClass<T>>
  • Doesn't play nice with other JVM languages (Scala, Kotlin)
  • Confusing for new users and contributors
  • Not consistent with all modules

Ideas

  • Using functional Builder-Pattern
    • Maybe Lombok @SuperBuilder?
  • Double Brace Initialization
    • People don't like this 😱

Double Brace Initialization 😜

private GenericContainer myContainer = new GenericContainer("myImage:42.23") {{
   withExposedPorts(4711);

   if (System.getenv().get("INSIDE_CI") == null) {
       org.testcontainers.Testcontainers.exposeHostPorts(8080);
   }

}};

Existing Kotlin 😱

val redisContainer = GenericContainer<Nothing>("redis:3-alpine")
                       .apply { 
                          withExposedPorts(6379) 
                       }

TDD workflows

  • Container startup introduces overhead
  • Testcontainers should be ephemeral
  • We still want to avoid test pollution and integrated tests

One more thing...

Welcome reusable containers!

Released in 1.12.3

Moar Future

  • Testcontainers 2.0
  • Complete Podman support
    • Working with RedHat on making the TC test suite feature parity reference test suite :)
  • Container-Core
    • Test-Framework agnostic
    • OO abstraction of Docker containers
  • Other languages
    • More languages and aligning existing forks
    • GraalVM (mind=blown)

Fire in OSS land 🔥

Fire in OSS land 🔥🔥

Peace restored 🤗

Far Future...

TODO: add Safe Harbor Statement here

  • Windows Containers on Windows (WCOW)
  • Orchestrators (Docker Swarm, Kubernetes)
    • Maybe one day... (Help from experts welcome!)
    • But what is the actual use case?
  • Other container engines (e.g. Podman, Firecracker)
  • Build and test inside containers with full IDE support
    • Cloud IDEs
    • Visual Studio Code Remote Development

GitHub Sponsors

Testcontainers - A Year in Review (JUG DO 2020)

By Kevin Wittek

Made with Slides.com

Testcontainers - A Year in Review (JUG DO 2020)

  • 842

More from Kevin Wittek