Intro to ES+CQRS
my story
- coupled code
- reusability via services
- inject dependencies
- thinking more about DB than about theproblem
RegistrationController
- validate data
- create user object
- persist user object
- create activation email
- send email
- save user to elasticsearch
RegistrationController
RegistrationService
ORM
RegistrationEmailManager
Mailer
EmailComposer
ORM
ElasticSearchManager
UserImportManager
UserCsvProcessor
UserValidationService
UserImportFetcher
ShoppingService
OrderService
UserManager
Product
Category
- id
- title
- description
- categoryId
- price
- vendorId
- createdAt
- id
- title
- parentId
- createdAt
Product
Category
- id
- title
- description
- categoryId
- category_level_0
- category_level_1
- category_level_2
- price
- vendorId
- createdAt
- id
- title
- parentId
- category_level_0
- category_level_1
- category_level_2
- createdAt
Product should be visible in all parent categories
Show products on site
class ProductsRepository
{
public function getProducts($categoryId,$categoryLevel,...){
...
$this->addWhere('category_level_'.$categoryLevel,$categoryId);
...
}
}
Edit category: change parent
- calculate new category levels
- find all products in that category
- update each category level
Pretty fine on 100 products
Timeout on 10.000 products
Product
Category
- id
- title
- description
- categoryId
- category_level_0
- category_level_1
- category_level_2
- discount
- price
- vendorId
- createdAt
- id
- title
- parentId
- category_level_0
- category_level_1
- category_level_2
- createdAt
We are opening special category with discounts
Show product on site
class ProductsRepository
{
public function getProducts($categoryId,$categoryLevel,...){
...
if( $categoryId === '34242'){
$this->addWhere('discount is not null');
}else{
$this->addWhere('category_level_'.$categoryLevel,$categoryId);
}
...
}
}
Get product price
class Product
{
public function getPrice(){
...
if( !is_null($this->discount)){
$price = $this->price;
$price *= (100 - $this->discount)/100;
return $price;
}
return $this->price;
}
}
Miro Svrtan
senior developer
@msvrtan
as web is growing
more and more core business systems are web
simple CMS
simple webshops
existing practices and pattern
existing practices and pattern
new ideas
eventsourcing
command query responsibility segregation
Domain
Our problem
web, cli, framework
Our DB, servers..
Infrastructure
Application
class RegisterController
{
public function registerUser($form){
$....
$this->validate(...);
$....
$this->fillUpUserObject(...);
$....
$this->save(...);
$....
$this->sendConfirmationEmail(...);
$this->saveUserInTextSearchEngine(...);
$....
$this->showView(...);
}
}
class RegisterController
{
public function registerUser($form){
$....
}
public function registerUserApi($apiData){
$....
}
}
class CsvImportController
{
public function importUsersCsv(){
$data = $this->readFromDisk('somefile.csv');
$....
}
}
class RemoteImportController
{
public function importFromRemoteSystem(){
$data = curl(..);
$....
}
}
something happened
do something else
event
subscriber
UserRegisteredEvent
SendActivationEmail
SaveInTextSearchEngine
class UserRegisteredEvent
{
private $userId;
public function __constuctor($userId)
{
$this->userId=$userId;
}
public function getUserId()
{
return $this->userId;
}
}
class SendActivationEmail
{
...
public function doWork(UserRegisteredEvent $event){
$....
}
...
}
class SaveUserInTextSearchEngine
{
...
public function doWork(UserRegisteredEvent $event){
$....
}
...
}
Message bus
How do we connect event to subcriber?
class MessageBus{
private $list=[];
public function add(string $eventName, array $subscriberNames)
{
$this->list[$eventName] = $subscriberNames;
}
public function doWork($event)
{
foreach($this->findSubscribers($command::class) as $subscriber)
{
$subscriber->doWork($event);
}
}
}
class RegisterController
{
public function registerUser($form){
$....
$this->validate(...);
$....
$this->fillUpUserObject(...);
$....
$this->save(...);
$....
$messagebus->doWork(new UserRegisteredEvent($userId));
$this->showView(...);
}
public function registerApiUser($json){
$....
$messagebus->doWork(new UserRegisteredEvent($userId));
$this->showView(...);
}
}
Changes
- MessageBus
- UserRegisteredEvent
- SendActivationEmail
- SaveInTextSearchEngine
Connect events->subscribers
we just registered a user
everything about sending email
everything about text search
How hard is it to modify now?
Switch email provider
- MessageBus
- UserRegisteredEvent
- SendActivationEmail
- SaveInTextSearchEngine
change this class
Change text search engine
- MessageBus
- UserRegisteredEvent
- SendActivationEmail
- SaveInTextSearchEngine
change this class
Stop sending activation emails
- MessageBus
- UserRegisteredEvent
SendActivationEmail- SaveInTextSearchEngine
delete this class
Registration
is still duplicated...
tell
do
command
handler
RegisterUserCommand
RegisterUserHandler
class RegisterUserCommand
{
private $userId;
private $username;
private $email;
public function __constuctor($userId,$username,$email)
{
$this->userId = $userId;
$this->username = $username;
$this->email = $email;
}
public function getUserId()
{
return $this->userId;
}
public function getUsername()
{
return $this->username;
}
public function getEmail()
{
return $this->email;
}
}
class RegisterUserHandler
{
...
public function handle(RegisterUserCommand $command)
{
$...
$...
$...
$messagebus->doWork(new UserRegisteredEvent($userId));
}
}
class RegisterController
{
public function registerUser($form){
$....
$this->validate(...);
$....
$commandbus->handle(new RegisterUserCommand(...));
$this->showView(...);
}
public function registerApiUser($json){
$....
$commandbus->handle(new RegisterUserCommand(...));
$this->showView(...);
}
}
CHE
command handler event
command pattern
event pattern
RegisterUserCommand
RegisterUserHandler
UserRegisteredEvent
SendActivationEmail
SaveToTextSearchEngine
RegistrationController
ImportController
CLI
AdminController
Commands
- action to take
- RegisterUser
- SendEmail
- BuyMilk
- command bus
- 1 command
- exactly 1 handler
Events
- what happened
- UserRegistered
- EmailSent
- MilkBought
- event bus
- 1 event
- 0,1,2,3,... subscribers
User still waits for email to be sent :(
- create SendActivationEmailCommand & Handler
- make SendActivationEmailSubscriber launch command instead of doing work
- add queueable command bus
- send command to queuable command bus
- worker(s) will do rest of the job
- eventual consistency issues!
Any questions on this part?
id | 123456789 |
---|---|
title | Register button doesnt work in IE |
assigned | null |
type | BUG |
status | OPEN |
priority | URGENT |
createdAt | 2016-06-11 22:22:11 |
updatedAt | 2017-04-18 11:01:35 |
closedAt |
Issue tracker: Issue
How long is this urgent bug opened?
No one is assigned to it?
Lets check the logs
- IssueOpened: 123456789 [2016-06-11 22:22:11]
- IssueClosed: 123456789 [2016-06-12 12:00:11]
- IssueReopened: 123456789, [2017-01-04 11:01:11]
- We are missing details now :(
- Lets add logging when priority changes
- 2months later: lets add logging when user is assigned/removed
New company policy: log everything
Until lazy developers forget to log something
We just lost valuable data :(
Logging stuff
class EventBus extends MessageBus
{
public function doWork($event)
{
parent::doWork($event);
$this->saveSomewhere($event);
}
}
id | event | createdAt |
---|---|---|
1 | O:19:"IssueOpened":3:{s:26:"IssueOpenedid";s:15:"123";s:26:"IssueOpenedname";s:5:"Title";..;} | 2010-03-17 11:22:33 |
2 | ... | ... |
... | ... | .. |
... | ... | .. |
... | O:17:"IssueTitleChanged":3:{s:26:"IssueTitleChanged";s:3:"124";s:4:"joe2";} | 2017-04-18 10:45 |
class UpdateIssueHandler
{
public function handle(UpdateIssue $command){
$....
$logger->info('Title on issue $id change to ' . $title);
}
}
class UpdateIssueHandler
{
public function handle(UpdateIssue $command){
$....
$eventbus->doWork(new IssueTitleChanged($id,$title));
}
}
I'm too lazy to add Event here
What if instead of logging, we create object state from events?
IssueCreated(123, 'Something', 'text', BUG, NORMAL, '2016..')
IssueAssigned(123, TeamManager, '2016-06-12 10:00:00')
PriorityChanged(123,URGENT, '2016-08-11 15:46:33')
IssueDellocated(123,'2016...')
IssueClosed(123,'2016...')
IssueReopened(123, '2017-04-18 11:01:35')
TitleChanged(123, 'Register button doesnt work in IE', '2016...')
Issue object is now in same state as in CRUD
Event sourcing
class Issue
{
private $id;
private $title;
private $text;
private $assigned;
private $type;
private $status;
private $priority;
public function changePriority($newPriority){
$this->priority = $newPriority;
}
}
CRUD
class Issue
{
...
public function changePriority($newPriority){
if($newPriority !== $this->priority){
$this->record(new PriorityChanged($this->id,$newPriority);
}
}
}
Eventsourcing #1
class Issue
{
...
private $newEvents=[];
public function changePriority($newPriority){
if($newPriority !== $this->priority){
$this->record(new PriorityChanged($this->id,$newPriority);
}
}
private function record($event){
$this->apply($event);
$this->newEvents[] = $event;
}
public function apply($event){
$className = get_class_name($event);
$methodName = apply.$className;
if( method_exists($this,$methodName){
$this->$methodName($event);
}
}
}
Eventsourcing #2
class Issue
{
...
private $newEvents=[];
public function changePriority($newPriority){
if($newPriority !== $this->priority){
$this->record(new PriorityChanged($this->id,$newPriority);
}
}
private function record($event){
$this->apply($event);
$this->newEvents[] = $event;
}
public function apply($event){
$className = get_class_name($event);
$methodName = apply.$className;
if( method_exists($this,$methodName){
$this->$methodName($event);
}
}
public function applyPriorityChanged(PriorityChanged $event){
$this->priority = $event->getPriority();
}
}
Eventsourcing #3
class Issue
{
...
private $newEvents=[];
public function changePriority($newPriority)
{
if($newPriority !== $this->priority){
$this->record(new PriorityChanged($this->id,$newPriority);
}
}
private function record($event)
{
$this->apply($event);
$this->newEvents[] = $event;
}
public function apply($event)
{
if( get_class_name($event) === PriorityChanged::class){
$this->priority = $event->getPriority();
}
..
}
}
Eventsourcing #4
$issue = new Issue();
$data = $db->get('SELECT * FROM issue WHERE id = 123');
foreach($data as $property => $value){
$propertySetterName = set.$property;
$issue->$propertySetterName($value);
}
Load Issue object: CRUD
$issue->changePriority('URGENT');
$db->update('UPDATE ...);
Save Issue object: CRUD
IssueCreated(123, 'Something', 'text', BUG, NORMAL, '2016..')
IssueAssigned(123, TeamManager, '2016-06-12 10:00:00')
PriorityChanged(123,URGENT, '2016-08-11 15:46:33')
IssueDellocated(123,'2016...')
IssueClosed(123,'2016...')
IssueReopened(123, '2017-04-18 11:01:35')
TitleChanged(123, 'Register button doesnt work in IE', '2016...')
id | class_name | object_id | event |
---|---|---|---|
... | |||
12 | Issue | 123 | serialized data |
13 | Issue | 123 | serialized data |
14 | User | 123 | serialized data |
15 | Issue | 92 | serialized data |
16 | User | 11 | serialized data |
17 | Issue | 323 | serialized data |
18 | Issue | 21 | serialized data |
... |
Event store table
$issue = new Issue();
$data = $db->get('SELECT * FROM event_store where object_id=123 and class_name=issue');
foreach($data as $item){
$event = deserialize($item['event']);
$event->apply($event);
}
Load Issue object: EventSourcing
$issue->changePriority('URGENT');
foreach( $issue->getNewEvents() as $newEvent)
{
$db->insert('INSERT INTO...');
}
Save Issue object: EventSourcing
Any questions on this part?
Hm, how do we show all open issues on our homepage?
We run events for each object and check state?
Hm, how do we show all open issues on our homepage?
We run events for each object and check state?
Any experience with text search engines?
- Our blog posts are in MySQL
- We save id, title and text to elasticsearch
- When user searches our blog, we ask elasticsearch for results
We want to make our blog posts text searchable and we use elastic search for that
$blogPost = new BlogPost();
$blogPost->setTitle(..)
...
$orm->save($blogPost);
$elasticSearch->save($id,$title,$text,...);
Saving a blog post
$results = $elasticSearch
->searchFor('some words');
Searching blog posts for 'some words'
CQRS
Command Query Responsibility Segregation
- separate models to write/read
- customized data for our need
IssueCreated(123, 'Something', 'text', BUG, NORMAL, '2016..')
IssueAssigned(123, TeamManager, '2016-06-12 10:00:00')
PriorityChanged(123,URGENT, '2016-08-11 15:46:33')
IssueDellocated(123,'2016...')
IssueClosed(123,'2016...')
IssueReopened(123, '2017-03-18 11:01:35')
TitleChanged(123, 'Register button doesnt work in IE', '2016...')
List of open issues: id and title
class OpenIssue
{
private $id;
private $title;
public function setId($title)
{
$this->id=$id;
}
public function getId()
{
return $this->id;
}
public function setTitle($title)
{
$this->title=$title;
}
public function getTitle()
{
return $this->title;
}
}
OpenIssue: CRUD
class IssueCreatedSubscriber
{
public function doWork(IssueCreated $event)
{
$openIssue = new OpenIssue();
$openIssue->setid($event->getId());
$openIssue->setTitle($event->getTitle());
$orm->save($openIssue);
}
}
class TitleChangedSubscriber
{
public function doWork(TitleChanged $event)
{
$openIssue = $orm->load($event->getId());
$openIssue->setTitle($event->getTitle());
$orm->save($openIssue);
}
}
class IssueClosedSubscriber
{
public function doWork(IssueClosed $event)
{
$openIssue = $orm->load($event->getId());
$orm->delete($openIssue);
}
}
List of open issues: id and title
OMG
Option 1: CRUD + LOG
Option 2: ES + CQRS + CRUD
quick
not losing data, scalable
CHE + CQRS example
Admin:Listings to verify
Listings to verify
Listings to verify
v2
ListingCreatedSubscriber
ListingRejectedSubscriber
EventSomethingHappenedSubscriber
...
v2
I didn't cover
- UUID
- DDD
- strategical
- tactical
- aggregates
- sagas/process managers
- popular frameworks
- broadway
- prooph
- message bus implementations
- SimpleBus
- Tactician
Quick recap
- intention instead of managers/processors/..
- domain/application/infrastructure separation
- try to solve next problem
- without web server, database or framework
- command /event patterns help decouple code
- events can be better than logging
- you can pick some ideas and use them today
Questions?
Feedback: https://joind.in/talk/37010
Thank you
Feedback: https://joind.in/talk/37010
Twitter: @msvrtan
Intro to ES+CQRS (PHP Serbia meetup #22 2017)
By Miro Svrtan
Intro to ES+CQRS (PHP Serbia meetup #22 2017)
Introduction to eventsourcing and command query responsibility segragation
- 1,998