Java Container Testing

Doctoral Researcher @ Institute for Internet Security
- Head of Blockchain Research Lab
Consultant @ Styrascosoft GbR
Testcontainers Maintainer and Open Source Enthusiast
Oracle Groundbreaker Ambassador
GitKraken Ambassador
Organizer Software Crafting Meetup Ruhr

Kevin Wittek @kiview

Key learnings

Easy setup of complete test environments
Not a lot effort to integration test using real external applications
Avoid integrated tests!

$ git clone https://github.com/foo/bar.git
$ cd bar
$ ./gradlew build

My definiton

For example:

Launch application server
Framework/Library interactions
Interact with external ports (e.g. network, file system, database)

"Tests which interact with external systems/dependencies"

But why a pyramid?

Source: https://labs.spotify.com/2018/01/11/testing-of-microservices/

Testing Honeycomb

Source: https://labs.spotify.com/2018/01/11/testing-of-microservices/, http://blog.thecodewhisperer.com/permalink/integrated-tests-are-a-scam

Integrated Tests

"I use the term integrated test to mean any test whose result (pass or fail) depends on the correctness of the implementation of more than one piece of non-trivial behavior." - J.B. Rainsberger

"A test that will pass or fail based on the correctness of another system." - Spotify

Integrated Tests

For example:

We (manually) spin up other services in a local testing environment
We test against other services in a shared testing environment
Changes to your system breaks tests for other systems

Blackbox

Only use public UI
- REST-API
- GUI
Startup complete application
SUT = Complete application and direct dependencies
Default: Outside of application process

Whitebox

Instantiate objects and collaborators
Might use TestDoubles
- Fakes, Mocks, Stubs, Spies, ...
Startup certain application slice or layer
SUT = Classes, Packages and Modules
Default: Inside of process

Solving the Integration Test Struggle

TestDoubles
Local DBs / Software
VMs (Vagrant)
Docker
Fig (-> Docker-Compose)
Docker API
- Testcontainers

But there is more!

Mutation Testing
- Testing the tests
Property Based Testing
- Generation of test cases
Explorative Session-Based Testing

Summary

Different kind of tests
Real integration tests instead of TestDoubles
Try to avoid integrated tests
Improve onboarding experience
Better portability of dev and test environment

Docker 101

Why Containers?

Development
Testing
Deployment
Operations
Playing & Fun!

# simple hello world
docker container run hello-world

# we can run applications and expose ports
docker container run -ti -p 8080:80 httpd
curl localhost:80

# containers are ephemeral
docker container run -ti busybox
> echo foo > bar.txt
> ls -la

docker container run -ti busybox

# there are different versions (tags) of images
docker container run --rm ubuntu:18.04 cat /etc/os-release
docker container run --rm ubuntu:19.10 cat /etc/os-release

Exercise: Docker Basics

Install Docker CE and run a hello-world container
- https://docs.docker.com/install/
Run a Tomcat container and curl it
- Use the Docker Hub to search for suitable images
Run a PostgreSQL container and connect to it

Containers vs VM

Kernel Space vs User Space

Docker Architecture

Image vs Container

Similar to Class vs Object
Image is a blue print
Container is the running instance
Image format
- <hostname>/<imageIdentifier>:<tag>
- Defaults for hostname and tag
- latest != latest

Building an image

FROM httpd:2.4
COPY ./public-html/ /usr/local/apache2/htdocs/

docker build -t myname:mytag .

Exercise: Image building

Build a webserver image (Apache)
Use an Ubuntu image as your base image

Volumes & Mounts

docker container run -ti -v /home/myuser:/home/dockeruser:ro ubuntu:latest


docker container run -v postgres_data:/var/lib/postgresql/data postgres
docker volume ls
docker volume inspect postgres_data

Volumes are persistent resources outside of containers
Use bind mounts to share host filesystem

Docker Networking

Expose and bind ports to map
Put Containers on isolated network
Use Dockers embedded DNS server

# create a Docker network
docker network create my-network

# start Apache webserver attached to this network
docker container run -itd --network=my-network --network-alias=apache httpd

# start empty Ubuntu container and interact with Apache container

docker container run -it --network=my-network ubuntu
> apt update && apt install -y curl
> curl apache

Docker-Compose

CLI tool
Define and manage 1-n containers
Declarative YAML

version: '2'

services:
  apache:
    image: httpd
    ports:
      - '80:80'

Exercise: Docker-Compose

Write a compose file for the Tomcat and your webserver image
- This is not a useful application, just for demo purposes
Make sure they both are accessible from localhost

Summary

Docker as a tool to run applications
Different Docker resources
Isolation
Reproducability
Codeable architecture

Welcome to Testcontainers

GenericContainer redis =
    new GenericContainer("redis:3.0.2")
               .withExposedPorts(6379);

redis.start();

// test my stuff

redis.stop();

The project

testcontainers-java first released in 2015
100% OSS, MIT licensed
65 releases, 149 contributors
Core maintainers
- Richard North
- Sergei Egorov
- Kevin Wittek
Forks in Python, C#, Rust, Go, JS; Scala wrapper

Capabilities

Generic docker container support
- use any Docker image to support tests
Databases (many!) and Stream processing (Kafka, Pulsar)
AWS mocks (Localstack)
Docker Compose
Selenium
Chaos testing

Capabilities (2)

Dynamic port binding and API
WaitStrategies
Docker environment discovery (e.g. docker-machine, DOCKER_HOST, Docker for Mac, Docker for Windows)
Platform independent
- Linux, macOS, Windows 10 (with NPIPE support!)

JDK12 Compatible!
+ regularly build against openjdk-ea

// Set up a redis container
@ClassRule
public static GenericContainer redis =
    new GenericContainer("redis:3.0.2")
               .withExposedPorts(6379);

JUnit4 Rules

Spock Extension


@Testcontainers
class TestContainersClassIT extends Specification {


    @Shared
    GenericContainer genericContainer = 
       new GenericContainer("postgres:latest")
            .withExposedPorts(5432)
            .withEnv([
                POSTGRES_USER: "foo"
                POSTGRES_PASSWORD: "secret"
            ])
}

Testcontainers-Jupiter (JUnit5)

@Testcontainers
class SomeTest {

    @Container
    private MySQLContainer mySQLContainer = new MySQLContainer();

    @Test
    void someTestMethod() {
        String url = mySQLContainer.getJdbcUrl();
         // create a connection and run test as normal
    }
}

Conference Voting App

Architecture

Exercise: Testcontainers

https://testcontainers.gitbook.io/workshop/

Exercise: Basic Smoke Test

Adding an abstract base integration test class
Test to check if context loads
Use SQL file to bootstrap database

Exercise: Integrate Testcontainers

Integration of Testcontainers using JDBC driver
Using real PostgreSQL database
Trying out different PostgreSQL version

Exercise: Healthcheck & RestAssured

RestAssured library for blackbox testing of web APIs
Spring-Boot Actuator endpoint
Checking healthiness of Spring-Boot Microservice

Exercise: Integration Redis

Using GenericContainer
Singleton-Container pattern using static initializer

Exercise: Blackbox API Test

Testing the application logic using the web API
Checking asynchronous behavior using Awaitility

Exercise: KafkaContainer

Integrate KafkaContainer
Special Testcontainers modules
Starting containers in paralell

Exercise: Run experiments

Whitebox test of RatingRepository
JUnit-Rule Integration
Tests as experiments

Summary

Testcontainers Java Library
JUnit-Rules or direct interaction
JdbcContainer
GenericContainer
KafkaContainer
Singleton-Container Pattern
RestAssured & Awaitility

Blackbox Testing

of JVM Applications

Test Drivers

Humans
Computers
- Scripts
- xUnit
- RestAssured
- Selenium WebDriver
- ...

Selenium WebDriver

Driver directly controls the Browser
- ChromeDriver
- FirefoxDriver
- InternetExplorerDriver
- ...
Language bindings to interact with browser specific driver
- Java
- C#
- Python
- ...

driver.get("http://www.google.com");
WebElement search = driver.findElement(By.name("q"));
search.sendKeys("testcontainers");
search.submit();

List<WebElement> results = new WebDriverWait(driver, 15)
  .until(ExpectedConditions.visibilityOfAllElementsLocatedBy(By.cssSelector("#search h3")));

assertTrue("the word 'testcontainers' appears in search results", 
  results.stream()
    .anyMatch(el -> el.getText().contains("testcontainers")));

Exposing Host

Summary

Browser automation with Selenium
Browser with Selenium in Container
Expose ports on host with Testcontainers
Record videos of Selenium tests
Manual debugging with VNC

System Testing Microservices


version: "3"

services:
  vote:
    image: vote-frontend
    command: python app.py
    volumes:
     - ./vote:/app
    ports:
      - "5000:80"

  redis:
    image: redis:alpine
    ports: ["6379"]

  worker:
    image: worker

  db:
    image: postgres:9.4

  result:
    image: result-frontend
    command: nodemon --debug server.js
    volumes:
      - ./result:/app
    ports:
      - "5001:80"
      - "5858:5858"

Exercise: Testing of non-JVM containers

Clone https://github.com/dockersamples/example-voting-app
Try out the microservice application
Perform a Selenium test using Testcontainers' DockerCompose support
- You need to add a Java Maven/Gradle project for this
- Test visiting of services and clicking the buttons
- For easier setup, use Testcontainers' expose host port feature together with docker-compose port binding

Summary

Reuse existing docker-compose files with DockerComposeContainer
Use familar Java libraries for testing of non JVM apps

CI inside containers?

Docker in Docker - DinD

Docker Wormhole

Future

and Common Pitfalls...

SELF-typing is sometimes a PITA

Doesn't play nice with other JVM languages (Scala, Kotlin)
Confusing for new users and contributors
Not consistent with all modules

Ideas

Using Builder-Pattern
- Maybe Lombok @SuperBuilder?
Double Brace Initialization
- People don't like this 😱

Double Brace Initialization 😜

private GenericContainer myContainer = new GenericContainer("myImage:42.23") {{
   withExposedPorts(4711);

   if (System.getenv().get("INSIDE_CI") == null) {
       org.testcontainers.Testcontainers.exposeHostPorts(8080);
   }

}};

TDD workflows

Container startup introduces overhead
Testcontainers should be ephemeral
We still want to avoid test pollution and integrated tests

Welcome reusable containers!

Future (2)

Testcontainers 2.0
Container-Core
- Test-Framework agnostic
- OO abstraction of Docker containers
Other languages
- More languages and aligning existing forks
- GraalVM (mind=blown)

Far Future...

TODO: add Safe Harbor Statement here

Windows Containers on Windows (WCOW)
Orchestrators (Docker Swarm, Kubernetes)
- Maybe one day... (Help from experts welcome!)
Other container engines (e.g. Podman)
Build and test inside containers with full IDE support
- Cloud IDEs
- Visual Studio Code Remote Development

Bonus Exercise: Hackergarten Session!

Sometimes the best way to learn a technology is working on it
Try to submit a real Github PR to the official Testcontainers project!
- Solve issues labeled with Good first issue
- Update documentation (Markdown in /docs)
- Develop a custom module (as a submodule)
- Contribute examples (in /examples subproject)

Java Container Testing

Kevin Wittek @kiview

Key learnings

My definiton

But why a pyramid?

Testing Honeycomb

Integrated Tests

Integrated Tests

Blackbox

Whitebox

Solving the Integration Test Struggle

But there is more!

Summary

Docker 101

Why Containers?

Exercise: Docker Basics

Containers vs VM

Kernel Space vs User Space

Docker Architecture

Image vs Container

Building an image

Exercise: Image building

Volumes & Mounts

Docker Networking

Docker-Compose

Exercise: Docker-Compose

Summary

Welcome to Testcontainers

The project

Capabilities

Capabilities (2)

JDK12 Compatible! + regularly build against openjdk-ea

JUnit4 Rules

Spock Extension

Testcontainers-Jupiter (JUnit5)

Conference Voting App

Architecture

Exercise: Testcontainers

Exercise: Basic Smoke Test

Exercise: Integrate Testcontainers

Exercise: Healthcheck & RestAssured

Exercise: Integration Redis

Exercise: Blackbox API Test

Exercise: KafkaContainer

Exercise: Run experiments

Summary

Blackbox Testing

of JVM Applications

Test Drivers

Selenium WebDriver

Exposing Host

Summary

System Testing Microservices

Exercise: Testing of non-JVM containers

Summary

CI inside containers?

Docker in Docker - DinD

Docker Wormhole

Future

and Common Pitfalls...

SELF-typing is sometimes a PITA

Ideas

Double Brace Initialization 😜

TDD workflows

Welcome reusable containers!

Future (2)

Far Future...

Bonus Exercise: Hackergarten Session!

Java Container Testing Workshop (JVM Con)

More from Kevin Wittek

JDK12 Compatible!
+ regularly build against openjdk-ea