Groovy Integration Testing

with Spock and Docker

Kevin Wittek

Software Developer @ GDATA Advanced Analytics

 

Consultant @ Styracosoft GbR

 

 

 

 

https://groovy-coder.com

         @Kiview

 

          https://github.com/kiview

Key learnings

  • Easy setup of complete test environments
  • Not a lot effort to integration test using real external applications
$ git clone https://github.com/foo/bar.git
$ cd bar
$ ./gradlew build

And what about Microservices?

My definiton

For example:

  • Launch application server
  • Framework/Library interactions
  • Interact with external ports (e.g. network, file system, database) 

"Tests which interact with external systems/dependencies"

Which tools to use?

(in the JVM world)

class HelloSpockSpec extends spock.lang.Specification {
  def "length of Spock's and his friends' names"() {
    expect:
    name.size() == length

    where:
    name     | length
    "Spock"  | 5
    "Kirk"   | 4
    "Scotty" | 6
  }
}  

Why combine?

  • Easy setup of dev environment
  • Uniform build and test environments
    • Also self contained and portable!
  • No installation and setup of external software
$ git clone https://github.com/foo/bar.git
$ cd bar
$ ./gradlew build

How to fuse both worlds?

Spock + Docker = Spocker?

@Docker(image = "emilevauge/whoami", ports = @PortBinding("8080:80"))
class DockerExtensionIT extends Specification {

    def "should start accessible docker container"() {
        given: "a http client"
        def client = HttpClientBuilder.create().build()

        when: "accessing web server"
        def response = client.execute(new HttpGet("http://localhost:8080"))

        then: "docker container is running and returns http status code 200"
        response.statusLine.statusCode == 200
    }

}

First commit

class DockerMethodInterceptor extends AbstractMethodInterceptor {

    private final DockerClientFacade dockerClient

    DockerMethodInterceptor(Docker docker) {
        this(new DockerClientFacade(docker))
    }

    DockerMethodInterceptor(DockerClientFacade dockerClient) {
        this.dockerClient = dockerClient
    }

    @Override
    void interceptSpecExecution(IMethodInvocation invocation) throws Throwable {
        dockerClient.startContainer()
        invocation.proceed()
        dockerClient.stopContainer()
    }
}

First commit

class DockerClientFacade {

    private final Docker imageConfig
    private final DockerClient dockerClient

    private static final Logger LOGGER = LoggerFactory.getLogger(DockerClientFacade)

    private String id

    DockerClientFacade(Docker containerConfig) {
        this.imageConfig = containerConfig
        dockerClient = initializeClient()
    }

    private static DockerClient initializeClient() {
        def clientConfig = DefaultDockerClientConfig.createDefaultConfigBuilder()
        return DockerClientBuilder.getInstance(clientConfig).build()
    }

    void startContainer() {
        def portBindings = parsePortBindings()
        ensureImageExists()
        id = createContainer(portBindings)
        dockerClient.startContainerCmd(id).exec()
    }

    private String createContainer(List<PortBinding> portBindings) {
        dockerClient.createContainerCmd(imageConfig.image())
                .withPortBindings(portBindings)
                .exec().id
    }

    private void ensureImageExists() {
        try {
            dockerClient.inspectImageCmd(imageConfig.image()).exec()
        } catch (NotFoundException e) {
            dockerClient.pullImageCmd(imageConfig.image()).exec(new PullImageResultCallback()).awaitSuccess()
        }
    }

    void stopContainer() {
        dockerClient.stopContainerCmd(id).exec()
        try {
            dockerClient.removeContainerCmd(id).exec()
        } catch(InternalServerErrorException e) {
            LOGGER.error("error while removing the container", e)
        }
    }

    List<PortBinding> parsePortBindings() {
        def ports = imageConfig.ports().first()

        return [parsePortBinding(ports)]
    }

    private PortBinding parsePortBinding(com.groovycoder.spockdockerextension.PortBinding ports) {
        def parsedBinding = PortBinding.parse(ports.value())
        return parsedBinding
    }
}

First commit

class DockerClientFacade {

    DockerClient dockerClient
    String image
    Map clientSpecificContainerConfig
    def containerStatus

    DockerClientFacade(Docker containerConfig) {
        image = containerConfig.image()
        dockerClient = new DockerClientImpl()
        clientSpecificContainerConfig = new DockerContainerConfigBuilder(containerConfig).build()
    }

    void startContainer() {
        containerStatus = dockerClient.run(image, clientSpecificContainerConfig, "latest")
    }

    void stopContainer() {
        def id = containerStatus.container.content.Id

        dockerClient.stop(id)
        dockerClient.rm(id)
    }

}

Switch to gesellix/docker-client

// Set up a redis container
@ClassRule
public static GenericContainer redis =
    new GenericContainer("redis:3.0.2")
               .withExposedPorts(6379);
  • Generic Containers
  • Specialized Containers (Databases, Selenium)
  • Dockerfile 
  • Dockerfile DSL
  • Docker-Compose
  • WaitStrategy
  • Docker environment discovery (e.g. docker-machine, DOCKER_HOST, ...)

Using testcontainers as docker-client

class DockerClientFacade {

    FixedHostPortGenericContainer dockerClient
    final String name
    final String[] ports
    final String image
    final Env[] env
    WaitStrategy waitStrategy

    DockerClientFacade(Docker containerConfig) {
        name = containerConfig.name()
        ports = containerConfig.ports()
        env = containerConfig.env()
        this.image = concatImageWithDefaultTagIfNeeded(containerConfig.image())
        waitStrategy = ((Closure) containerConfig.waitStrategy().newInstance(this, this))()
    }

    private static String concatImageWithDefaultTagIfNeeded(String image) {
        (image.contains(":")) ? image : image + ":latest"
    }

    void run() {
        try {
            start()
        } catch (ContainerLaunchException e) {
            throw new DockerRunException("Error running the container", e)

        }
    }

    void rm() {
        dockerClient.stop()
    }

    void start() {
        dockerClient = new FixedHostPortGenericContainer(image)
        ports.each { String portMapping ->
            def split = portMapping.split(":")
            dockerClient.withFixedExposedPort(split[0].toInteger(), split[1].toInteger())
        }

        env.each { Env e ->
            dockerClient.withEnv(e.key(), e.value())
        }

        dockerClient.waitingFor(waitStrategy)

        dockerClient.start()
    }

    void stop() {
        dockerClient.stop()
    }

    String getIp() {
        //noinspection GrDeprecatedAPIUsage
        dockerClient.containerInfo.networkSettings.ipAddress
    }

}

Annotations a good idea?

@Docker(image = "postgres", ports = ["5432:5432"], env = [
        @Env(key = "POSTGRES_USER", value = "foo"),
        @Env(key = "POSTGRES_PASSWORD", value = "secret")
        ], 
        waitStrategy = { new JdbcWaitStrategy(username: "foo", 
            password: "secret", 
            jdbcUrl: "jdbc:postgresql:foo", 
            driverClassName: "org.postgresql.Driver", 
            testQueryString: "SELECT 1") })
def "waits until postgres accepts jdbc connections"() {
    // tests
}

Fully embrace testcontainers


@Testcontainers
class TestContainersClassIT extends Specification {

    WaitStrategy postgresWaitStrategy = new JdbcWaitStrategy(
        username: "foo", 
        password: "secret", 
        jdbcUrl: "jdbc:postgresql:foo", 
        driverClassName: "org.postgresql.Driver", 
        testQueryString: "SELECT 1")

    @Shared
    GenericContainer genericContainer = new GenericContainer("postgres:latest")
            .withExposedPorts(5432)
            .withEnv([
                POSTGRES_USER: "foo"
                POSTGRES_PASSWORD: "secret"
            ])

Demo!

Docker-Compose


version: "3"

services:
  vote:
    image: vote-frontend
    command: python app.py
    volumes:
     - ./vote:/app
    ports:
      - "5000:80"

  redis:
    image: redis:alpine
    ports: ["6379"]

  worker:
    image: worker

  db:
    image: postgres:9.4

  result:
    image: result-frontend
    command: nodemon --debug server.js
    volumes:
      - ./result:/app
    ports:
      - "5001:80"
      - "5858:5858"
class GebHomepageSpec extends GebSpec {
    def "can access The Book of Geb via homepage"() {
        when:
        to GebHomePage

        and:
        highlights.jQueryLikeApi.click()

        then:
        sectionTitles == ["Navigating Content", "Form Control Shortcuts"]
        highlights.jQueryLikeApi.selected
    }
}

Moar Demo!

CI inside containers?

Docker in Docker - DinD

Docker Wormhole

Docker Wormhole

Future

  • Better support for Docker networking
  • Improve Windows support
  • Build and test inside containers with full IDE support

Resources

Integration testing with Spock and Docker

By Kevin Wittek

Integration testing with Spock and Docker

  • 4,414