Gaining Confidence

Frederik Hahne

JUG Bielefeld

February 12th, 2019

About Me

Frederik Hahne

@atomfrede

Software Developer @wescalehq

@java_hipster board member

@jugpaderborn organizer

@devoxx4kids mentor

Gaining Confidence

Context

  • a lot of (micro)services
  • api first
  • diverse stack
https://www.martinfowler.com/articles/microservice-testing/#conclusion-test-pyramid
https://labs.spotify.com/2018/01/11/testing-of-microservices/

Integration Testing

  • Do not rely on the correctness of other services
  • Database (Mocks)
  • Service Mocks
  • API/Contracts

But still

Give us confidence that the code does what it should.

Provide feedback that is fast, accurate, reliable and predictable.

https://labs.spotify.com/2018/01/11/testing-of-microservices/

Databases
Services
APIs

Databases

  • Mock/Stub Database
  • in memory database
  • mongo
  • solr, elastic
  • jsonb in psql

Testcontainers

  • Database Containers
  • Selenium Containers
  • Kafka, Elastic, ...
  • Generic Containers
  • Docker Compose
  • Dockerfiles
@Autowired
public JooqProductRepository(DSLContext jooq) {
  this.jooq = jooq;
}

@Override
public Optional<Product> findOneById(Long id) {
       
  return jooq.selectFrom(PRODUCT_TABLE)
         .where(PRODUCT_TABLE.ID.eq(id))
         .fetchOptional()
         .map(record -> recordMapper.toProduct(record));
}
@JooqTest
class JooqProductRepositoryIT extends Specification {

  @Autowired
  DSLContext jooq

  def 'fetch non existing product by id'() {

    given:
    def subject = new JooqProductRepository(jooq)

    when:
    def result = subject.findOneById(4711)

    then:
    result != null
    result.isEmpty()
  }
}

Demo


spring.datasource.url=jdbc:tc:postgresql:11.0://localhost:5432/confidence
spring.datasource.driver-class-name=org.testcontainers.jdbc.ContainerDatabaseDriver

@Testcontainers
class JooqProductRepositoryIT extends Specification {

  @Shared
  PostgreSQLContainer postgreSQLContainer = new PostgreSQLContainer()
    .withDatabaseName("confidence")
    .withUsername("confidence")
    .withPassword("confidence")

  @Shared
  DataSource datasource

  def setupSpec() {

    HikariConfig config = new HikariConfig()
    config.setUsername(postgreSQLContainer.username)
    config.setJdbcUrl(postgreSQLContainer.jdbcUrl)
    config.setPassword(postgreSQLContainer.password)
    datasource = new HikariDataSource(config)

    Flyway flyway = new Flyway().configure()
      .dataSource(datasource)
      .load()
    flyway.migrate()
  }

 def '...'() {
    
   given:
   def subject = new JooqProductRepository(DSL.using(datasource, SQLDialect.POSTGRES))
 }

Title Text

There is more...

  • Testing interaction with other services
  • Wrap call into an adapter
  • Mock the adapter
  • Returns POJOs
  • Test your logic

Not Good Enough

  • De/Serialization
  • Handling of Web Exceptions
  • Caches, Retries, ...
  @Rule
  WireMockRule wireMockRule = new WireMockRule(options().dynamicPort())

  def "should load ratings"() {

    given:
    def subject = new ProductRatingConnector(new OkHttpClientFactory(), 
                      "http://localhost:${wireMockRule.port()}")
    and:
    //language=json
    def successBody = '''[
  {
    "id": 1,
    "title": "Nice Product",
    "rating": 5,
    "description": "Lorem Ipsum",
    "productId": 1
  }
]
'''
    and:
    stubFor(get(urlPathEqualTo("/reviews"))
      .withQueryParam("productId", equalTo("1"))
      .willReturn(aResponse()
      .withStatus(200)
      .withBody(successBody)))

    when:
    def result = subject.fetchProductRatings(1)

    then:
    result != null
    result.size() == 1

  }
  static GenericContainer hoverfly = new GenericContainer("spectolabs/hoverfly")
    .withCommand("-webserver")
    .withExposedPorts(8888, 8500)

  @BeforeAll
  static void setUp() {
    hoverfly.start()
    setupSimulation()
  }
  static setupSimulation() {
    HoverflyClient hoverflyClient = HoverflyClient.custom()
      .host(hoverfly.getContainerIpAddress()).port(hoverfly.getMappedPort(8888)).build()
    hoverflyClient.setMode(HoverflyMode.SIMULATE)
    hoverflyClient.setSimulation(dsl(simulateServer(),).getSimulation())
  }
  static simulateServer() {
    //language=json
    def successBody = '''[
  {
    "id": 1,
    "title": "Nice Product",
    "rating": 5,
    "description": "Lorem Ipsum",
    "productId": 1
  }
]'''
    return service("http://${hoverfly.getContainerIpAddress()}:${hoverfly.getMappedPort(8500)}")
      .get("reviews")
      .queryParam("productId", RequestFieldMatcher.newExactMatcher("1"))
      .anyQueryParams().willReturn(success().body(successBody))
  }

  @Test
  void shouldLoadRatings() {
    def subject = new ProductRatingConnector(new OkHttpClientFactory(), 
                    "http://${hoverfly.getContainerIpAddress()}:${hoverfly.getMappedPort(8500)}/")
    def result = subject.fetchProductRatings(1)
    assertThat(result).isNotEmpty()
  }

APIs

  • OpenAPI
  • Works as expected
  • Breaking Changes
  • Implementation adhere to spec

How?

  • Spring Boot Test
  • REST Assured
  • Swagger Request Validator

A Java library for validating HTTP request/responses against an OpenAPI / Swagger specification.

https://bitbucket.org/atlassian/swagger-request-validator
  TestFilters(String swaggerFileLocation) {

    def file = new File(swaggerFileLocation)

    def map = file.withReader {
      Yaml yaml = new Yaml()
      return yaml.load(it)
    }

    def whitelist = ValidationErrorsWhitelist.create()
      .withRule("Ignore 'application/problem+json' does not match any allowed types.",
      allOf(messageContains("Response Content-Type header 'application/problem\\+json' does not match any allowed types.*")))

    def interactionValidator = OpenApiInteractionValidator
      .createFor(new JSONObject(map).toString())
      .withWhitelist(whitelist)
      .build()

    swaggerValidationFilter = new OpenApiValidationFilter(interactionValidator)
  }
@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
class ProductsApiIT {

  @LocalServerPort
  private int port

  @BeforeAll
  static void setup() throws IOException {

    RestAssured.reset()

    def testFilters = new TestFilters(TestPaths.resolve("../../../api-spec/server/products.yaml").toString())

    RestAssured.filters(
      testFilters.swaggerValidationFilter
    )
    RestAssured.enableLoggingOfRequestAndResponseIfValidationFails()
  }

  @BeforeEach
  void before() throws IOException {
    RestAssured.baseURI = "http://localhost:${port}/v1"
  }
  @Test
  void create_product() {

    //language=json
    def productCreate = '''
{
  "name": "Thinkpad T480s",
  "description": "A thinkpad, as simple as that"
}
'''

    given()
    .contentType(ContentType.JSON)
    .body(productCreate)
    .when()
    .post("/products")
    .then().statusCode(201)

  }

Request/Response Validation

  • Works your implementation as expected?
  • Does the request match the specification?
  • Does the response and codes match the specification?

Where are the Containers?

@ClassRule
static PostgreSQLContainer postgreSQLContainer = new PostgreSQLContainer("postgres:11.0")
    .withDatabaseName("confidence")
    .withUsername("confidence")
    .withPassword("confidence")
    .withNetwork(Network.SHARED)
    .withNetworkAliases("postgres")

static GenericContainer bootJar

@BeforeClass
static void setup() {

    bootJar = new GenericContainer(new ImageFromDockerfile().withDockerfileFromBuilder(
      { builder ->
        builder.from("openjdk:11-jdk-stretch")
          .volume("/tmp")
          .copy("confidence-0.0.1-SNAPSHOT.jar", "app.jar")
          .entryPoint("java", "-Dspring.datasource.password=confidence",
          "-Dspring.datasource.url=jdbc:postgresql:postgres:5432/confidence",
          "-Djava.security.egd=file:/dev/./urandom", "-jar", "/app.jar")
          .build()
      })
      .withFileFromPath("confidence-0.0.1-SNAPSHOT.jar", 
            TestPaths.resolve("../../../build/libs/confidence-0.0.1-SNAPSHOT.jar")))
      .withExposedPorts(8080)
      .withNetwork(postgreSQLContainer.getNetwork())

    bootJar.start()
}

@Before
void clear() {
    RestAssured.baseURI = "http://${bootJar.getContainerIpAddress()}:${bootJar.getMappedPort(8080)}/v1"
}

Testcontainers helped us

  • to test apps that where hard to test
  • to protect our apis against regressions
  • have fun with integration tests
  • but be careful no to forget unit tests and testable design!

Keep your tests

Outlook

Questions?

Resources

  • https://slides.com/kiview/testcontainers-whs
  • https://github.com/testcontainers/testcontainers-java
  • https://gitlab.com/atomfrede/confidence
  • https://slides.com/kiview/testcontainers-intro

The serverless framework

Mittwoch, 20.02.2019

18.00 Uhr

R2DBC

Mittwoch, 03.04.2019

18.00 Uhr

Gaining Confidence

By atomfrede

Gaining Confidence

Slides for Gaining Confidence Talk at JUG Bielefeld in Feb. 2019

  • 1,561