CQRS + EVENT SOURCING IN NODEJS
BY ROMAN
ABOUT ME
Software engineer and team lead at DA-14
CONTENTS
Theory
How did we choose it?
How it supposed to work?
Problems and Solutions
What would I recommend instead?
Other
THEORY FOR BEGINNERS: CQRS AND EVENT SOURCING
THEORY FOR BEGINNERS: CQRS AND EVENT SOURCING
Command Query Responsibility Segregation (CQRS)
THEORY FOR BEGINNERS: CQRS AND EVENT SOURCING
Command Query Responsibility Segregation (CQRS)
THEORY FOR BEGINNERS: CQRS AND EVENT SOURCING
Command Query Responsibility Segregation (CQRS)
THEORY FOR BEGINNERS: CQRS AND EVENT SOURCING
Command Query Responsibility Segregation (CQRS)
resources
product
warehouse
production
factory
THEORY FOR BEGINNERS: CQRS AND EVENT SOURCING
Command Query Responsibility Segregation (CQRS)
resources
product
warehouse
production
factory
Huge Load
THEORY FOR BEGINNERS: CQRS AND EVENT SOURCING
Command Query Responsibility Segregation (CQRS)
balancer
THEORY FOR BEGINNERS: CQRS AND EVENT SOURCING
Command Query Responsibility Segregation (CQRS)
THEORY FOR BEGINNERS: CQRS AND EVENT SOURCING
Command Query Responsibility Segregation (CQRS)
THEORY FOR BEGINNERS: CQRS AND EVENT SOURCING
Command Query Responsibility Segregation (CQRS)
balancer
THEORY FOR BEGINNERS: CQRS AND EVENT SOURCING
CQRS vs typical CRUD: benefits
THEORY FOR BEGINNERS: CQRS AND EVENT SOURCING
Event Sourcing
THEORY FOR BEGINNERS: CQRS AND EVENT SOURCING
Event Sourcing
start
finish
You, good guy
villian
villian
villian
THEORY FOR BEGINNERS: CQRS AND EVENT SOURCING
Event Sourcing
Voldemort
Aid Kit
Hannibal
Palpatine
Aid Kit
Start
Warehouse
THEORY FOR BEGINNERS: CQRS AND EVENT SOURCING
Event Sourcing
Voldemort
Aid Kit
Hannibal
Palpatine
Aid Kit
Start
Warehouse
THEORY FOR BEGINNERS: CQRS AND EVENT SOURCING
Event Sourcing
Voldemort
Aid Kit
Hannibal
Palpatine
Aid Kit
Start
Warehouse
THEORY FOR BEGINNERS: CQRS AND EVENT SOURCING
Event Sourcing
Voldemort
Aid Kit
Hannibal
Palpatine
Aid Kit
Start
Warehouse
THEORY FOR BEGINNERS: CQRS AND EVENT SOURCING
Event Sourcing
Voldemort
Aid Kit
Hannibal
Palpatine
Aid Kit
Start
Warehouse
THEORY FOR BEGINNERS: CQRS AND EVENT SOURCING
Event Sourcing
Voldemort
Aid Kit
Hannibal
Palpatine
Aid Kit
Start
Warehouse
Warehouse
Steps
THEORY FOR BEGINNERS: CQRS AND EVENT SOURCING
Event Sourcing
Voldemort
Aid Kit
Hannibal
Palpatine
Aid Kit
Start
Steps
Voldemort: fight
Warehouse
Warehouse
THEORY FOR BEGINNERS: CQRS AND EVENT SOURCING
Event Sourcing
Voldemort
Aid Kit
Hannibal
Palpatine
Aid Kit
Voldemort: fight
Voldemort: passed
Start
Warehouse
Warehouse
Steps
THEORY FOR BEGINNERS: CQRS AND EVENT SOURCING
Event Sourcing
Voldemort
Aid Kit #1
Hannibal
Palpatine
Aid Kit #2
Start
Voldemort: passed
Aid Kid#1: used
Steps
Voldemort: fight
Warehouse
THEORY FOR BEGINNERS: CQRS AND EVENT SOURCING
Event Sourcing
Voldemort
Aid Kit #1
Hannibal
Palpatine
Aid Kit #2
Start
Hannibal: fight
Aid Kit#1: used
Voldemort: passed
Voldemort: fight
Steps
Warehouse
THEORY FOR BEGINNERS: CQRS AND EVENT SOURCING
Event Sourcing
Voldemort
Aid Kit #1
Hannibal
Palpatine
Aid Kit #2
Start
Hannibal: failed
Aid Kit#1: used
Voldemort: passed
Voldemort: fight
Steps
Hannibal: fight
Warehouse
THEORY FOR BEGINNERS: CQRS AND EVENT SOURCING
Event Sourcing
Voldemort
Aid Kit #1
Hannibal
Palpatine
Aid Kit #2
Start
Warehouse
Hannibal: fight
Aid Kit#1: used
Voldemort: passed
Warehouse
Voldemort: fight
Steps
THEORY FOR BEGINNERS: CQRS AND EVENT SOURCING
Voldemort: passed
Aid Kit#1: used
Hannibal: fight
Hannibal: failed
Event Sourcing
Voldemort: fight
Warehouse
States or events results
THEORY FOR BEGINNERS: CQRS AND EVENT SOURCING
Event Sourcing
Hannibal: fight
shoot 3 times
THEORY FOR BEGINNERS: CQRS AND EVENT SOURCING
Event Sourcing
Hannibal: fight
event: shoot 3 times => event: { amo: { use: 3 } };
THEORY FOR BEGINNERS: CQRS AND EVENT SOURCING
Event Sourcing
Initial snapshot
Current state
LIVES: 0
AMO: 0
POINTS: 0
LIVES: 0
AMO: 0
POINTS: 0
THEORY FOR BEGINNERS: CQRS AND EVENT SOURCING
Event Sourcing
Warehouse
{ amo: { load: 200, use: 0 }, lives: { add: 100, deduct: 0 }, addPoints: 0 }
Initial snapshot
Current state
LIVES: 100
AMO: 200
POINTS: 0
LIVES: 0
AMO: 0
POINTS: 0
THEORY FOR BEGINNERS: CQRS AND EVENT SOURCING
Event Sourcing
Warehouse
Voldemort: fight
{ amo: { load: 200, use: 0 }, lives: { add: 100, deduct: 0 }, addPoints: 0 }
{ amo: { load: 0, use: 3 }, lives: { add: 0, deduct: 0 }, addPoints: 0 }
Current state
LIVES: 100
AMO: 197
POINTS: 0
THEORY FOR BEGINNERS: CQRS AND EVENT SOURCING
Event Sourcing
Voldemort: fight
Voldemort: passed
{ amo: { load: 200, use: 0 }, lives: { add: 100, deduct: 0 }, addPoints: 0 }
{ amo: { load: 0, use: 3 }, lives: { add: 0, deduct: 0 }, addPoints: 0 }
{ amo: { load: 0, use: 10 }, lives: { add: 0, deduct: 25 }, addPoints: 10 }
Warehouse
Current state
LIVES: 75
AMO: 187
POINTS: 10
THEORY FOR BEGINNERS: CQRS AND EVENT SOURCING
Event Sourcing
Voldemort: fight
Voldemort: passed
{ amo: { load: 200, use: 0 }, lives: { add: 100, deduct: 0 }, addPoints: 0 }
{ amo: { load: 0, use: 3 }, lives: { add: 0, deduct: 0 }, addPoints: 0 }
{ amo: { load: 0, use: 10 }, lives: { add: 0, deduct: 25 }, addPoints: 10 }
Warehouse
{ amo: { load: 0, use: 0 }, lives: { add: 15, deduct: 0 }, addPoints: 0 }
Aid Kit#1: used
Current state
LIVES: 90
AMO: 187
POINTS: 10
THEORY FOR BEGINNERS: CQRS AND EVENT SOURCING
Hannibal: fight
Hannibal: failed
Event Sourcing
Voldemort: fight
Voldemort: passed
{ amo: { load: 200, use: 0 }, lives: { add: 100, deduct: 0 }, addPoints: 0 }
{ amo: { load: 0, use: 3 }, lives: { add: 0, deduct: 0 }, addPoints: 0 }
{ amo: { load: 0, use: 10 }, lives: { add: 0, deduct: 25 }, addPoints: 10 }
Warehouse
{ amo: { load: 0, use: 0 }, lives: { add: 15, deduct: 0 }, addPoints: 0 }
Aid Kit#1: used
{ amo: { load: 0, use: 57 }, lives: { add: 0, deduct: 0 }, addPoints: 0 }
{ amo: { load: 0, use: 5 }, lives: { add: 0, deduct: 120 }, addPoints: 0 }
Current state
LIVES: -30
AMO: 125
POINTS: 10
THEORY FOR BEGINNERS: CQRS AND EVENT SOURCING
Event Sourcing
Events
Consequences
Hannibal: fight
Hannibal: failed
Voldemort: fight
Voldemort: passed
Warehouse
Aid Kit#1: used
{ amo: { load: 200, use: 0 }, lives: { add: 100, deduct: 0 }, addPoints: 0 }
{ amo: { load: 0, use: 3 }, lives: { add: 0, deduct: 0 }, addPoints: 0 }
{ amo: { load: 0, use: 10 }, lives: { add: 0, deduct: 25 }, addPoints: 10 }
{ amo: { load: 0, use: 0 }, lives: { add: 15, deduct: 0 }, addPoints: 0 }
{ amo: { load: 0, use: 57 }, lives: { add: 0, deduct: 0 }, addPoints: 0 }
{ amo: { load: 0, use: 5 }, lives: { add: 0, deduct: 120 }, addPoints: 0 }
THEORY FOR BEGINNERS: CQRS AND EVENT SOURCING
Event Sourcing: benefits
HOW DID WE CHOOSE THIS?
Conditions
HOW DID WE CHOOSE THIS?
Conditions
HOW DID WE CHOOSE THIS?
Conditions
HOW DID WE CHOOSE THIS?
Conditions
HOW DID WE CHOOSE THIS?
HOW IT SUPPOSED TO WORK?
Architecture Items
HOW IT SUPPOSED TO WORK?
Client
API
Queries
Commands
MS0
MS1
MSn
Denormalizer
Event Bus
Event store
HOW IT SUPPOSED TO WORK?
Stack
HOW IT SUPPOSED TO WORK?
Stack
HOW IT SUPPOSED TO WORK?
Stack
HOW IT SUPPOSED TO WORK?
Stack
HOW IT SUPPOSED TO WORK?
commands
queries
HOW IT SUPPOSED TO WORK?
PROBLEMS: #0 HARD TO UPDATE FUNCTIONALITY
PROBLEMS: #0 HARD TO UPDATE FUNCTIONALITY
Problem
API gateway
Service (X)
Denormalizer
DB Model (X)
DB Model (X)
DB Model (X)
PROBLEMS: #0 HARD TO UPDATE FUNCTIONALITY
Solution
API gateway
Service (X)
Denormalizer
DB Model (X) Repo
PROBLEMS: #1 COULDN'T FIND SUITABLE LIBRARIES
PROBLEMS: #1 COULDN'T FIND SUITABLE LIBRARIES
Problem
PROBLEMS: #1 COULDN'T FIND SUITABLE LIBRARIES
Solution: make it easy - write a script
Service
Deployment script
Build
Kafka topics creation script
PROBLEMS: #2 HEALTHCHECK
PROBLEMS: #2 HEALTHCHECK
Simple schema
DOWN
UP
UP
HTTP API Gateway
Command
Success response
PROBLEMS: #2 HEALTHCHECK
Problem
PROBLEMS: #2 HEALTHCHECK
DOWN
UP
UP
HTTP API Gateway
Command
Success response
Solution: Simple HTTP request is enough
HTTP
GET /status
HTTP
GET /status
HTTP
GET /status
PROBLEMS: #3 ERROR HANDLING
PROBLEMS: #3 ERROR HANDLING
Problem: there is no way to notify a client about errors due to the event-driven architecture
MS0
ERROR
Command
Success response
HTTP API Gateway
MS1
MSn
PROBLEMS: #3 ERROR HANDLING
Service
ERROR
Notification Service
PROBLEMS: #4 LOGGING
PROBLEMS: #4 LOGGING
Regular Service 0
Regular Service 1
Regular Service n
Loggin Service
HTTP Access
SSH Access
PROBLEMS: #4 LOGGING
Solution
PROBLEMS: #5 GROWING EVENT LOG
PROBLEMS: #5 GROWING EVENT LOG
Timeline
PROBLEMS: #5 GROWING EVENT LOG
Timeline
PROBLEMS: #5 GROWING EVENT LOG
Timeline
PROBLEMS: #5 GROWING EVENT LOG
Timeline
PROBLEMS: #5 GROWING EVENT LOG
Solution: make snapshots
Event Store
Snapshot Store
event (n)
event (10)
...
event (9)
event (8)
event (6)
event (5)
event (4)
event (3)
event (2)
event (1)
snapshot (1)
snapshot (2)
snapshot (3)
snapshot (n)
...
...
...
...
...
...
...
PROBLEMS: #5 GROWING EVENT LOG
Solution
But
WHAT WOULD I RECOMMEND INSTEAD?
WHAT WOULD I RECOMMEND INSTEAD? (IN MY CASE)
WHEN IT'S RECOMMENDED TO USE CQRS + EVENT SOURCING?
WHEN IT'S RECOMMENDED TO USE CQRS + EVENT SOURCING?
BE AWARE OF
BE AWARE OF
CQRS + Event Sourcing Cons
OFFTOP BUT PAY ATTENTION
OFFTOP BUT PAY ATTENTION
Event Sourcing is all about functional programming
BEST PRACTICES
BEST PRACTICES
HELPFUL MODULES
HELPFUL MODULES
ARTICLES
ARTICLES
CQRS
Event Sourcing
CQRS + Event Sourcing
QUESTIONS
*and again I couldn't find a funny meme for this page
roman.sachenko@gmail.com
https://www.facebook.com/rsachenko
https://github.com/roman-sachenko
https://twitter.com/RSachenko
roman.sachenko