Testcontainers

The Past, the Present and the Future

  • ​Blockchain Research Group Lead @ Institute for Internet Security
    • PhD Student @ RWTH Aachen
  • Consultant & Coach
  • Testcontainers Maintainer and Open Source Enthusiast
  • Blockchain Activities
    • ​Bloxberg Consortium
    • Sovrin Steward Council
  • Oracle Groundbreaker Ambassador

Kevin Wittek          @kiview

Thanks to Sergei!

(for slides, idea and everything else)

@bsideup

Source: https://watirmelon.blog/2012/01/31/introducing-the-software-testing-ice-cream-cone/

Source: https://labs.spotify.com/2018/01/11/testing-of-microservices/

Testing Honeycomb

GenericContainer redis =
    new GenericContainer("redis:3.0.2")
               .withExposedPorts(6379);

redis.start();

// test my stuff

redis.stop();

The Project

  • testcontainers-java first released in 2015
  • 100% OSS, MIT licensed
  • 65 releases, 149 contributors
  • Core maintainers
    • Richard North
    • Sergei Egorov
    • Kevin Wittek
  • Forks in Python, C#, Rust, Go, JS; Scala wrapper

@whichrich

Capabilities

  • Generic docker container support
    • use any Docker image to support tests
  • Databases (many!) and Stream processing (Kafka, Pulsar)
  • AWS mocks (Localstack)
  • Docker Compose
  • Selenium
  • Chaos testing

Capabilities (2)

  • Dynamic port binding and API
  • WaitStrategies
  • Docker environment discovery (e.g. docker-machine, DOCKER_HOST, Docker for Mac, Docker for Windows)
  • Platform independent
    • Linux, macOS, Windows 10 (with NPIPE support!)

Users

Testcontainers-Jupiter (JUnit5)

@Testcontainers
class SomeTest {

    @Container
    private MySQLContainer mySQLContainer = new MySQLContainer();

    @Test
    void someTestMethod() {
        String url = mySQLContainer.getJdbcUrl();
         // create a connection and run test as normal
    }
}

Demo?

DOs

  • Use Testcontainers
  • Use copyFileToContainer

DONT's

  • Use fixed host ports
  • Disable Ryuk without understanding the implications
  • Assume Podman is a Docker drop-in replacement

What happened so far...

Sep, 2018 (1.9.x)

  • OkHttp by default
  • Windows npipe support
  • Registry auth support on Windows
  • Fix local compose on Windows
  • Host ports exposing
Testcontainers.exposeHostPorts(localServerPort);
final String rootUrl =
    String.format("http://host.testcontainers.internal:%d/", localServerPort);

Sep, 2018 (1.9.x)

  • Dynamic port binding in Couchbase module
  • New modules
    • ClickHouse
    • PostGIS

Nov, 2018 (1.10.x)

  • JUnit5 (Jupiter) support
  • Documentation relaunch
    • Code examples from as part of CI
  • ENV var to turn off Ryuk (TESTCONTAINERS_RYUK_DISABLED=true)
  • shm + tmpfs settings
  • Dependabot
  • New modules
    • Neo4j & Elasticsearch
You can see an example test that could 
have been written for it (without using Testcontainers):

<!--codeinclude-->
[Pre-Testcontainers test code]
(../examples/junit5/redis/src/test/java/quickstart/RedisBackedCacheIntTestStep0.java) 
block:RedisBackedCacheIntTestStep0
<!--/codeinclude-->

Dependabot PRs

Dependabot FTW!

Mar, 2019 (1.11.x)

  • Chaos testing support (toxiproxy)
  • fsync=off for PostgreSQL module
  • Drop Netty transport
  • Reworking shading

Jul, 2019 (1.12.x)

  • dependsOn API
  • Improved pull handling
  • CI for Windows!
    • Azure Pipelines with private Windows node
  • New modules
    • DB2, CockroachDB & RabbitMQ
GenericContainer container = new GenericContainer()
        .dependsOn(postgresContainer, kafkaContainer);

Jul, 2019 (1.12.x)

  • Secure logging defaults for properties and environment variables
  • Use official Oracle JDBC driver
  • Support for DockerClient event streaming
  • Docker Compose --build support
  • Image pull policies (AgeBasedPullPolicy) 
new GenericContainer<>(imageName)
        .withImagePullPolicy(PullPolicy.alwaysPull())

AgeBasedPullPolicy oneHour = new AgeBasedPullPolicy(
  Duration.of(1L, ChronoUnit.HOURS)
);
new GenericContainer<>(imageName)
        .withImagePullPolicy(oneHour)

Mar, 2020 (1.13.x)

  • Access Docker Compose containers by service name
  • New modules
    • Presto & OrientDB
public DockerComposeContainer environment = 
  new DockerComposeContainer(new File("src/test/resources/compose-test.yml"))
        .withExposedService("redis_1", REDIS_PORT)
        .withExposedService("db_1", 3306);

Optional<ContainerState> result = environment.getContainerByServiceName("db_1");

Apr, 2020 (1.14.x)

  • docker-java updated to 3.2.x (deprecations!)
  • R2DBC support
  • Improved Couchbase support
  • JUnit Jupiter and Spock is now TestLifecycleAware
    • Supporting VNC recording
r2dbc:tc:mysql:///databasename?TC_IMAGE_TAG=5.7.22

Apr, 2020 (1.14.x)

Oct, 2020 (1.15.x RC)

  • Support for rootless Docker!
  • Deprecation of ambiguous constructors
  • Un-shade docker-java-api
  • Optional Apache HttpClinet 5 transport
  • GCloud module
  • Confluent 6 and MongoDB 4.4 support
  • Docker for Mac fixes

// deprecrate
new XyzContainer()
new XyzContainer(String)

// in favor for
new XyzContainer(DockerImageName)

// allowing
new XyzContainer( DockerImageName.parse( "the/image:tag" ) )

Oct, 2020 (1.15.x RC)

  • Image compatibility check, less magic!
// will just work
new KafkaContainer(DockerImageName.parse("confluentinc/cp-kafka:any"));

// will initially fail
new KafkaContainer(DockerImageName.parse("some-other-kafka"));

// be explicit instead!
new KafkaContainer(DockerImageName.parse("some-other-kafka"))
  .asCompatibleSubstituteFor("confluentinc/cp-kafka");

Future

and Common Pitfalls...

SELF-typing is sometimes a PITA

  • Pattern defined in "Effective Java"
    • Generic type with a recursive type paramter
    • MyClass<T extends MyClass<T>>
  • Doesn't play nice with other JVM languages (Scala, Kotlin)
  • Confusing for new users and contributors
  • Not consistent with all modules

Ideas

  • Using functional Builder-Pattern
    • Maybe Lombok @SuperBuilder?
  • Double Brace Initialization
    • People don't like this 😱

Double Brace Initialization 😜

private GenericContainer myContainer = new GenericContainer("myImage:42.23") {{
   withExposedPorts(4711);

   if (System.getenv().get("INSIDE_CI") == null) {
       org.testcontainers.Testcontainers.exposeHostPorts(8080);
   }

}};

Existing Kotlin 😱

val redisContainer = GenericContainer<Nothing>("redis:3-alpine")
                       .apply { 
                          withExposedPorts(6379) 
                       }

TDD workflows

  • Container startup introduces overhead
  • Testcontainers should be ephemeral
  • We still want to avoid test pollution and integrated tests

One more thing...

Welcome reusable containers!

Released in 1.12.3

Moar Future

  • Testcontainers 2.0
  • Complete Podman support
    • Working with RedHat on making the TC test suite feature parity reference test suite :)
  • Container-Core
    • Test-Framework agnostic
    • OO abstraction of Docker containers
  • Other languages
    • More languages and aligning existing forks
    • GraalVM (mind=blown)

Fire in OSS land 🔥

Fire in OSS land 🔥🔥

Peace restored 🤗

Far Future...

TODO: add Safe Harbor Statement here

  • Windows Containers on Windows (WCOW)
  • Orchestrators (Docker Swarm, Kubernetes)
    • Maybe one day... (Help from experts welcome!)
    • But what is the actual use case?
  • Other container engines (e.g. Podman, Firecracker)
  • Build and test inside containers with full IDE support
    • Cloud IDEs
    • Visual Studio Code Remote Development

GitHub Sponsors

Testcontainers - The Past, the Present and the Future (JCON 2020)

By Kevin Wittek

Made with Slides.com

Testcontainers - The Past, the Present and the Future (JCON 2020)

  • 921

More from Kevin Wittek