Liran Tal
💚@nodejs Security WG member | Core team @meanjs, #dockly | Author: 📘Essential Node.js Security ✨ Engineering Manager @nielsen-oss | ❤️#opensource #javascript
A better approach for API Testing
Consumer-Driven Contracts
Liran Tal
github.com/lirantal
@liran_tal
Developer Advocate at Snyk
Node.js Security WG
author of Essential Node.js Security
Liran Tal
github.com/lirantal
@liran_tal
Developer Advocate at Snyk
A QA Engineer
walks into a bar...
Orders a beer
A QA Engineer
walks into a bar...
Orders 0 beers
A QA Engineer
walks into a bar...
Orders 999999 beers
A QA Engineer
walks into a bar...
Orders <script>alert('1beer')</script>
A QA Engineer
walks into a bar...
Unit
Testing
E2E
Testing
UA
Testing
Security
Testing
A11ly
Testing
Perf
Testing
Tomcat
MySQL
Monoliths
Build
Test
Deploy
- Simple processes
- One team
- Easy business domain evolution
- Straight-forward testing
Build
Test
Deploy
Build
Test
Deploy
Build
Test
Deploy
Microservices
- Orchestration
- Monitoring
- Configuration
Microservices
in reality
src: https://apigee.com/about/blog/api-technology/microservices-amazon
Microservices
Testing ?
Testing Microservices
Option 1 - Mocks
Build
Test
Deploy
Build
Test
Deploy
GET /stats
Team A
Team B
GET /stats
mock API calls
Testing Microservices
Option 1 - Mocks
Cons
Pros
1. Not reliable
1. Quick
2. Cheap
3. Fast
4. Deterministic
Testing Microservices
Build
Test
Deploy
GET /stats
Team A
Team B
Option 2 - End-to-End
Team C
Testing Microservices
Cons
Pros
1. Slow
2. Costly
3. Not scalable
1. Reliable
Option 2 - End-to-End
Testing Microservices
Option 2 - End-to-End
Slow
Cost
Testing Microservices
Mocks or End-to-End...
Are these my only choices?
🤔
GET /stats
data: {
"movieId": 1001,
"name": "The Matrix"
"rating": 10
}
200 OK
What needs to be tested?
The Request
The Response
The Payload
Contract Testing Heaven
Consumer-Driven Contracts
Consumer-Driven Contracts
Integration Testing at Scale
2-Way API Contract Enforcement
API Documentation + Mock
Language SDKs
Contract Testing Framework
Open Source
response: {
"status": 200
"body": []
}
Consumer
Contract
Provider
Pact Broker
Collaborate
Consumer-Driven Contracts
Collaborate
Consumer-Driven Contracts
Collaborate
Write
Contract Tests
Consumer-Driven Contracts
Collaborate
Write
Contract Tests
request: {
"method": "GET",
"path": "/projects"
},
response: {
"status": 200
"body": []
}
Consumer-Driven Contracts
Collaborate
Push Contract to Broker
Write
Contract Tests
Consumer-Driven Contracts
Collaborate
Push Contract to Broker
Write
Contract Tests
Pact Broker
Consumer-Driven Contracts
Collaborate
Push Contract to Broker
Write
Contract Tests
Provider
Implements
Consumer-Driven Contracts
Collaborate
Push Contract to Broker
Write
Contract Tests
Provider
Implements
function amazing_api_endpoint() {
// incredible logic goes here
// ...
// ...
}Consumer-Driven Contracts
Collaborate
Push Contract to Broker
Write
Contract Tests
Provider
Implements
Download
Contracts
Consumer-Driven Contracts
Collaborate
Push Contract to Broker
Write
Contract Tests
Download
Contracts
Provider
Implements
request: {
"method": "GET",
"path": "/projects"
},
response: {
"status": 200
"body": []
}
Consumer-Driven Contracts
Collaborate
Push Contract to Broker
Write
Contract Tests
Download
Contracts
Provider
Implements
Run Provider
Contract Tests
Consumer-Driven Contracts
Consumer-Driven Contracts
Collaborate
Push Contract to Broker
Write
Contract Tests
Download
Contracts
Provider
Implements
Run Provider
Contract Tests
request: {
"method": "GET",
"path": "/projects"
},
response: {
"status": 200
"body": []
}
Users
Service
Get Contracts
Create Contract
Consumer-Driven Contracts
Projects
Service
Validate Contract Not Broken
Use Contract
Mock in Tests
Let's build something...
YouMDB
Serverless-side Rendering
VR
AR
IMIU-AR
Async Flooks
Blockchain
Web: ^2.0.0 technology
YouMDB
Consumer
Movies
Provider
Reviews
YouMDB
YouMDB
Consumer
Movies
Provider
Reviews
data: {
"movieId": 1001,
"name": "The Matrix"
"totalReviews": 10,
"averageRating": 8
}
GET /movies
?
data: {
"movieId": 1001,
"name": "The Matrix"
"totalReviews": 10,
"averageRating": 8
}
GET /stats
Consumer Testing
Client > Client Mock > Contract Test >
HTTP Client
Tests?
Unit Test it ofcourse!
Sinon? Nock?
Import HTTP Client Import Pact
Declare new mocked provider
Spin it up
Client > Client Mock > Contract Test >
Define interaction
Expected
request payload
Expected
response payload
Client > Client Mock > Contract Test >
Trigger the client
Verify expectations
Client > Client Mock > Contract Test >
Client > Client Mock > Contract Test >
Consumer Guidelines
1. Interactions are in test cases
2. Consider using factories for interactions
3. Avoid random data in expectations
4. Match types instead of actual data
5. Don't tempt into E2E functional tests
response: {
"status": 200
"body": []
}
Contract
Client > Client Mock > Contract Test >
What is a contract?
{
interaction
consumer provider
request & response
*postel's law
Client > Client Mock > Contract Test >
What is a contract?
Pact Broker
Pact Contract
Pact Broker
Consumer
Provider
Pact Broker
Pact Broker
-
See The Contract
Pact Broker
-
Consult verification status
-
Built-in API Explorer powered by HAL
-
Verified Version Matrix overview
Provider
Specify the
consumer-provider
Where to download the contract from
Provider > Contract Test
Start verifying
Endpoint to control
states
For each state transition the DB accordingly
Provider > State Management
State Management 🏋️♀️
GET /stats
data: {
"movieId": 1001,
"name": "The Matrix"
"rating": 10
}
200 OK
The Request
State Management 🏋️♀️
Provider
Reviews
POST /setup
1. API Service starts...
2. API Service /setup
{
state:
"has no stats"
}
GET /stats
200 []
Consumer-Driven Contracts
Collaborate
Push Contract to Broker
Write
Contract Tests
Download
Contracts
Provider
Implements
Run Provider
Contract Tests
Bad Practices
🙅♀️
1. Don't test Provider’s Business Logic
2. Using Pact as a mock service
3. Contracts for Public APIs
Benefits of CDC
🤗
1. Waste reduction
2. Win both Confidence and Speed
3. True release independence
Stay Calm
and
Embrace
Consumer-Driven Contracts
Liran Tal
github.com/lirantal
@liran_tal
Developer Advocate at Snyk
APIdays Barcelona - Consumer-Driven Contracts for API Testing
By Liran Tal
APIdays Barcelona - Consumer-Driven Contracts for API Testing
A Comprehensive Guide to Contract Testing APIs in a Service Oriented Architecture. It is likely you experienced the painful situation of deploying to production only to find out that an API service you integrate with has broken the contract. Enter Consumer-Driven-Contracts.
