Gaining Confidence
Frederik Hahne
JUG Bielefeld
February 12th, 2019
About Me
Frederik Hahne
@atomfrede
Software Developer @wescalehq
@java_hipster board member
@jugpaderborn organizer
@devoxx4kids mentor
Gaining Confidence
Context
- a lot of (micro)services
- api first
- diverse stack
https://www.martinfowler.com/articles/microservice-testing/#conclusion-test-pyramid
https://labs.spotify.com/2018/01/11/testing-of-microservices/
Integration Testing
- Do not rely on the correctness of other services
- Database (Mocks)
- Service Mocks
- API/Contracts
But still
Give us confidence that the code does what it should.
Provide feedback that is fast, accurate, reliable and predictable.
https://labs.spotify.com/2018/01/11/testing-of-microservices/
Databases
Services
APIs
Databases
- Mock/Stub Database
- in memory database
- mongo
- solr, elastic
- jsonb in psql
Testcontainers
- Database Containers
- Selenium Containers
- Kafka, Elastic, ...
- Generic Containers
- Docker Compose
- Dockerfiles
@Autowired
public JooqProductRepository(DSLContext jooq) {
this.jooq = jooq;
}
@Override
public Optional<Product> findOneById(Long id) {
return jooq.selectFrom(PRODUCT_TABLE)
.where(PRODUCT_TABLE.ID.eq(id))
.fetchOptional()
.map(record -> recordMapper.toProduct(record));
}
@JooqTest
class JooqProductRepositoryIT extends Specification {
@Autowired
DSLContext jooq
def 'fetch non existing product by id'() {
given:
def subject = new JooqProductRepository(jooq)
when:
def result = subject.findOneById(4711)
then:
result != null
result.isEmpty()
}
}
Demo
spring.datasource.url=jdbc:tc:postgresql:11.0://localhost:5432/confidence
spring.datasource.driver-class-name=org.testcontainers.jdbc.ContainerDatabaseDriver
@Testcontainers
class JooqProductRepositoryIT extends Specification {
@Shared
PostgreSQLContainer postgreSQLContainer = new PostgreSQLContainer()
.withDatabaseName("confidence")
.withUsername("confidence")
.withPassword("confidence")
@Shared
DataSource datasource
def setupSpec() {
HikariConfig config = new HikariConfig()
config.setUsername(postgreSQLContainer.username)
config.setJdbcUrl(postgreSQLContainer.jdbcUrl)
config.setPassword(postgreSQLContainer.password)
datasource = new HikariDataSource(config)
Flyway flyway = new Flyway().configure()
.dataSource(datasource)
.load()
flyway.migrate()
}
def '...'() {
given:
def subject = new JooqProductRepository(DSL.using(datasource, SQLDialect.POSTGRES))
}
Title Text
There is more...
- Testing interaction with other services
- Wrap call into an adapter
- Mock the adapter
- Returns POJOs
- Test your logic
Not Good Enough
- De/Serialization
- Handling of Web Exceptions
- Caches, Retries, ...
@Rule
WireMockRule wireMockRule = new WireMockRule(options().dynamicPort())
def "should load ratings"() {
given:
def subject = new ProductRatingConnector(new OkHttpClientFactory(),
"http://localhost:${wireMockRule.port()}")
and:
//language=json
def successBody = '''[
{
"id": 1,
"title": "Nice Product",
"rating": 5,
"description": "Lorem Ipsum",
"productId": 1
}
]
'''
and:
stubFor(get(urlPathEqualTo("/reviews"))
.withQueryParam("productId", equalTo("1"))
.willReturn(aResponse()
.withStatus(200)
.withBody(successBody)))
when:
def result = subject.fetchProductRatings(1)
then:
result != null
result.size() == 1
}
static GenericContainer hoverfly = new GenericContainer("spectolabs/hoverfly")
.withCommand("-webserver")
.withExposedPorts(8888, 8500)
@BeforeAll
static void setUp() {
hoverfly.start()
setupSimulation()
}
static setupSimulation() {
HoverflyClient hoverflyClient = HoverflyClient.custom()
.host(hoverfly.getContainerIpAddress()).port(hoverfly.getMappedPort(8888)).build()
hoverflyClient.setMode(HoverflyMode.SIMULATE)
hoverflyClient.setSimulation(dsl(simulateServer(),).getSimulation())
}
static simulateServer() {
//language=json
def successBody = '''[
{
"id": 1,
"title": "Nice Product",
"rating": 5,
"description": "Lorem Ipsum",
"productId": 1
}
]'''
return service("http://${hoverfly.getContainerIpAddress()}:${hoverfly.getMappedPort(8500)}")
.get("reviews")
.queryParam("productId", RequestFieldMatcher.newExactMatcher("1"))
.anyQueryParams().willReturn(success().body(successBody))
}
@Test
void shouldLoadRatings() {
def subject = new ProductRatingConnector(new OkHttpClientFactory(),
"http://${hoverfly.getContainerIpAddress()}:${hoverfly.getMappedPort(8500)}/")
def result = subject.fetchProductRatings(1)
assertThat(result).isNotEmpty()
}
APIs
- OpenAPI
- Works as expected
- Breaking Changes
- Implementation adhere to spec
How?
- Spring Boot Test
- REST Assured
- Swagger Request Validator
A Java library for validating HTTP request/responses against an OpenAPI / Swagger specification.
https://bitbucket.org/atlassian/swagger-request-validator
TestFilters(String swaggerFileLocation) {
def file = new File(swaggerFileLocation)
def map = file.withReader {
Yaml yaml = new Yaml()
return yaml.load(it)
}
def whitelist = ValidationErrorsWhitelist.create()
.withRule("Ignore 'application/problem+json' does not match any allowed types.",
allOf(messageContains("Response Content-Type header 'application/problem\\+json' does not match any allowed types.*")))
def interactionValidator = OpenApiInteractionValidator
.createFor(new JSONObject(map).toString())
.withWhitelist(whitelist)
.build()
swaggerValidationFilter = new OpenApiValidationFilter(interactionValidator)
}
@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
class ProductsApiIT {
@LocalServerPort
private int port
@BeforeAll
static void setup() throws IOException {
RestAssured.reset()
def testFilters = new TestFilters(TestPaths.resolve("../../../api-spec/server/products.yaml").toString())
RestAssured.filters(
testFilters.swaggerValidationFilter
)
RestAssured.enableLoggingOfRequestAndResponseIfValidationFails()
}
@BeforeEach
void before() throws IOException {
RestAssured.baseURI = "http://localhost:${port}/v1"
}
@Test
void create_product() {
//language=json
def productCreate = '''
{
"name": "Thinkpad T480s",
"description": "A thinkpad, as simple as that"
}
'''
given()
.contentType(ContentType.JSON)
.body(productCreate)
.when()
.post("/products")
.then().statusCode(201)
}
Request/Response Validation
- Works your implementation as expected?
- Does the request match the specification?
- Does the response and codes match the specification?
Where are the Containers?
@ClassRule
static PostgreSQLContainer postgreSQLContainer = new PostgreSQLContainer("postgres:11.0")
.withDatabaseName("confidence")
.withUsername("confidence")
.withPassword("confidence")
.withNetwork(Network.SHARED)
.withNetworkAliases("postgres")
static GenericContainer bootJar
@BeforeClass
static void setup() {
bootJar = new GenericContainer(new ImageFromDockerfile().withDockerfileFromBuilder(
{ builder ->
builder.from("openjdk:11-jdk-stretch")
.volume("/tmp")
.copy("confidence-0.0.1-SNAPSHOT.jar", "app.jar")
.entryPoint("java", "-Dspring.datasource.password=confidence",
"-Dspring.datasource.url=jdbc:postgresql:postgres:5432/confidence",
"-Djava.security.egd=file:/dev/./urandom", "-jar", "/app.jar")
.build()
})
.withFileFromPath("confidence-0.0.1-SNAPSHOT.jar",
TestPaths.resolve("../../../build/libs/confidence-0.0.1-SNAPSHOT.jar")))
.withExposedPorts(8080)
.withNetwork(postgreSQLContainer.getNetwork())
bootJar.start()
}
@Before
void clear() {
RestAssured.baseURI = "http://${bootJar.getContainerIpAddress()}:${bootJar.getMappedPort(8080)}/v1"
}
Testcontainers helped us
- to test apps that where hard to test
- to protect our apis against regressions
- have fun with integration tests
- but be careful no to forget unit tests and testable design!
Keep your tests
Outlook
Questions?
Resources
- https://slides.com/kiview/testcontainers-whs
- https://github.com/testcontainers/testcontainers-java
- https://gitlab.com/atomfrede/confidence
- https://slides.com/kiview/testcontainers-intro
The serverless framework
Mittwoch, 20.02.2019
18.00 Uhr
R2DBC
Mittwoch, 03.04.2019
18.00 Uhr
Gaining Confidence
By atomfrede
Gaining Confidence
Slides for Gaining Confidence Talk at JUG Bielefeld in Feb. 2019
- 1,748