In-house Circuit Breaker
A Presentation
by Ronak Raithatha
1. Circuit Breaker Pattern
2. Opossum Library
3. Tripp
A pattern to improve the resiliency of your application by encapsulating failure-prone dependencies with robust fallback options
Failure-prone dependencies:
- External API calls
- Database connections
- Fragile inter-service operations
const CircuitBreaker = require('opossum');
function asyncFunctionThatCouldFail (x, y) {
return new Promise((resolve, reject) => {
// Do something, maybe on the network or a disk
});
}
const options = {
timeout: 3000, // If our function takes longer than 3 seconds, trigger a failure
errorThresholdPercentage: 50, // When 50% of requests fail, trip the circuit
resetTimeout: 30000 // After 30 seconds, try again.
};
const breaker = new CircuitBreaker(asyncFunctionThatCouldFail, options);
// if asyncFunctionThatCouldFail starts to fail, firing the breaker
// will trigger our fallback function
breaker.fallback(() => 'Sorry, out of service right now');
breaker.fire(params)
.then(console.log)
.catch(console.error);
const circuit = new CircuitBreaker(() => axios.get(route), circuitBreakerOptions);
circuit.fallback(() => ({ body: `${route} unavailable right now. Try later.` }));
circuit.on('success',
(result) => console.log(`SUCCESS: ${JSON.stringify(result)}`));
circuit.on('timeout',
() => console.log(`TIMEOUT: ${route} is taking too long to respond.`));
circuit.on('reject',
() => console.log(`REJECTED: The breaker for ${route} is open. Failing fast.`));
circuit.on('open',
() => console.log(`OPEN: The breaker for ${route} just opened.`));
circuit.on('halfOpen',
() => console.log(`HALF_OPEN: The breaker for ${route} is half open.`));
circuit.on('close',
() => console.log(`CLOSE: The breaker for ${route} has closed. Service OK.`));
circuit.on('fallback',
(data) => console.log(`FALLBACK: ${JSON.stringify(data)}`));
Fairly simple API for what it achieves
For the most part, this fit our needs, but we decided to go bigger.
Create something that can be used across multiple streams.
Common logging pattern
Ability to get metrics without custom code from each stream
Modularize how features were added and shared among streams
Create a separation between CB and external call being made
Is a set of tooling encapsulating Opossum
Extensibility through Redux-like middleware API
Built-in wrappers for Apollo and Axios to promote good DX by exposing underlying API.
Made with ❤️ by MyAudi
This is Tripp* of house MyAudi,
the First in Audi-toolbox,
Breaker of Circuits,
Preserver of Good DX and
Propagator of Extensibility Through Robust Middleware API
* Now available in a Toolbox near you