Intro to EventSourcing & CQRS

my story

RegistrationController

validate data
create user object
persist user object
create activation email
send email
save user to elasticsearch

RegistrationController

RegistrationService

ORM

RegistrationEmailManager

Mailer

EmailComposer

ORM

ElasticSearchManager

UserImportManager

UserCsvProcessor

UserValidationService

UserImportFetcher

ShoppingService

OrderService

UserManager

Product

Edit category: change parent

Pretty fine on 100 products

Timeout on 10.000 products

Product

Miro Svrtan

senior engineer / trainer

@msvrtan

as web is growing

more and more core business systems are web

simple CMS

simple webshops

existing practices and pattern

new ideas

eventsourcing

command query responsibility segregation

Domain

Our problem

web, cli

framework, DB, servers..

Infrastructure

Application

class RegisterController
{
    public function registerUser($form){
       $....
       $this->validate(...);
       $....
       $this->fillUpUserObject(...);
       $....
       $this->save(...);
       $....
       $this->sendConfirmationEmail(...);
       $this->saveUserInTextSearchEngine(...);
       $....
       $this->showView(...);
    }
}

class RegisterController
{
    public function registerUser($form){
       $....
    }
    public function registerUserApi($apiData){
       $....
    }
}

class CsvImportController
{
    public function importUsersCsv(){
       $data = $this->readFromDisk('somefile.csv');
       $....
    }
}


class RemoteImportController
{
    public function importFromRemoteSystem(){
       $data = curl(..);
       $....
    }
}

something happened

do something else

event

subscriber

UserRegisteredEvent

SendActivationEmail

SaveInTextSearchEngine

class UserRegisteredEvent
{
    private $userId;

    public function __constuctor($userId)
    {
        $this->userId=$userId;
    }

    public function getUserId()
    {
       return $this->userId;
    }
}

class SendActivationEmail
{
    ... 

    public function doWork(UserRegisteredEvent $event){
       $....
    }

    ...
}

class SaveUserInTextSearchEngine
{
    ...

    public function doWork(UserRegisteredEvent $event){
       $....
    }

    ...
}

Message bus

How do we connect event to subcriber?


class MessageBus{

    private $list=[];

    public function add(string $eventName, array $subscriberNames)
    {
        $this->list[$eventName] = $subscriberNames;
    }

    public function doWork($event)
    {
        foreach($this->findSubscribers($event::class) as $subscriber)
        {
             $subscriber->doWork($event);
        }
    }
}

class RegisterController
{
    public function registerUser($form){
       $....
       $this->validate(...);
       $....
       $this->fillUpUserObject(...);
       $....
       $this->save(...);
       $....
       $messagebus->doWork(new UserRegisteredEvent($userId));
       $this->showView(...);
    }

    public function registerApiUser($json){
       $....
       $messagebus->doWork(new UserRegisteredEvent($userId));
       $this->showView(...);
    }
}

Changes

MessageBus
UserRegisteredEvent
SendActivationEmail
SaveInTextSearchEngine

Connect events->subscribers

we just registered a user

everything about sending email

everything about text search

How hard is it to modify now?

Switch email provider

MessageBus
UserRegisteredEvent
SendActivationEmail
SaveInTextSearchEngine

change this class

Change text search engine

MessageBus
UserRegisteredEvent
SendActivationEmail
SaveInTextSearchEngine

change this class

Stop sending activation emails

MessageBus
UserRegisteredEvent
~~SendActivationEmail~~
SaveInTextSearchEngine

delete this class

Registration

is still duplicated...

tell

command

handler

RegisterUserCommand

RegisterUserHandler

class RegisterUserCommand
{
    private $userId;
    private $username;
    private $email;

    public function __constuctor($userId,$username,$email)
    {
        $this->userId    = $userId;
        $this->username = $username;
        $this->email    = $email;
    }

    public function getUserId()
    {
       return $this->userId;
    }

    public function getUsername()
    {
       return $this->username;
    }

    public function getEmail()
    {
       return $this->email;
    }
}

class RegisterUserHandler
{
    ...

    public function handle(RegisterUserCommand $command)
    {
        $...
        $...
        $...
        $messagebus->doWork(new UserRegisteredEvent($userId));
    }

}

class RegisterController
{
    public function registerUser($form){
       $....
       $this->validate(...);
       $....
       $commandbus->handle(new RegisterUserCommand(...));
       $this->showView(...);
    }

    public function registerApiUser($json){
       $....
       $commandbus->handle(new RegisterUserCommand(...));
       $this->showView(...);
    }
}

CHE

command handler event

command pattern

event pattern

RegisterUserCommand

RegisterUserHandler

UserRegisteredEvent

SendActivationEmail

SaveToTextSearchEngine

RegistrationController

ImportController

CLI

AdminController

Commands

action to take
- RegisterUser
- SendEmail
- BuyMilk
command bus
1 command
- exactly 1 handler

Events

what happened
- UserRegistered
- EmailSent
- MilkBought
event bus
1 event
- 0,1,2,3,... subscribers

User still waits for email to be sent :(

create SendActivationEmailCommand & Handler
make SendActivationEmailSubscriber launch command instead of doing work
add queueable command bus

Any questions on this part?

id	123456789
title	Register button doesnt work in IE
assigned	null
type	BUG
status	OPEN
priority	URGENT
createdAt	2016-06-11 22:22:11
updatedAt	2018-09-13 11:01:35
closedAt

Issue tracker: Issue

How long is this urgent bug opened?

No one is assigned to it?

Lets check the logs

IssueOpened: 123456789 [2016-06-11 22:22:11]

IssueClosed: 123456789 [2016-06-12 12:00:11]

We are missing details now :(
Lets add logging when priority changes
2months later: lets add logging when user is assigned/removed

New company policy: log everything

Until lazy developers forget to log something

We just lost valuable data :(

Logging stuff


class EventBus extends MessageBus
{

    public function doWork($event)
    {
        parent::doWork($event);
        $this->saveSomewhere($event);
    }
}

id	event	createdAt
1	O:19:"IssueOpened":3:{s:26:"IssueOpenedid";s:15:"123";s:26:"IssueOpenedname";s:5:"Title";..;}	2010-03-17 11:22:33
2	...	...
...	...	..
...	...	..
...	O:17:"IssueTitleChanged":3:{s:26:"IssueTitleChanged";s:3:"124";s:4:"joe2";}	2018-09-13 10:45

class UpdateIssueHandler
{
    public function handle(UpdateIssue $command){
       $....
       $logger->info('Title on issue $id change to ' . $title);
    }
}

class UpdateIssueHandler
{
    public function handle(UpdateIssue $command){
       $....
       $eventbus->doWork(new IssueTitleChanged($id,$title));
    }
}

I'm too lazy to add Event here

What if instead of logging, we create object state from events?

IssueCreated(123, 'Something', 'text', BUG, NORMAL, '2016..')

IssueAssigned(123, TeamManager, '2016-06-12 10:00:00')

PriorityChanged(123,URGENT, '2016-08-11 15:46:33')

IssueDellocated(123,'2016...')

IssueClosed(123,'2016...')

IssueReopened(123, '2018-09-13 11:01:35')

TitleChanged(123, 'Register button doesnt work in IE', '2016...')

Issue object is now in same state as in CRUD

Event sourcing

class Issue
{
    private $id;
    private $title;
    private $text;
    private $assigned;
    private $type;
    private $status;
    private $priority;

    public function changePriority($newPriority){
        $this->priority = $newPriority;
    }

}

CRUD

class Issue
{
    ...

    public function changePriority($newPriority){
        if($newPriority !== $this->priority){
            $this->record(new PriorityChanged($this->id,$newPriority);
        }
    }
}

Eventsourcing: change priority

class Issue
{
    ...

    private $newEvents=[];


    public function changePriority($newPriority){
        if($newPriority !== $this->priority){
            $this->record(new PriorityChanged($this->id,$newPriority);
        }
    }

    private function record($event){
        $this->apply($event);
        $this->newEvents[] = $event;
    }

    public function apply($event){
        $className = get_class_name($event);
        $methodName = apply.$className;
        if( method_exists($this,$methodName){
             $this->$methodName($event);
        }
    }
}

Eventsourcing: record & apply

class Issue
{
    ...

    private $newEvents=[];


    public function changePriority($newPriority){
        if($newPriority !== $this->priority){
            $this->record(new PriorityChanged($this->id,$newPriority);
        }
    }

    private function record($event){
        $this->apply($event);
        $this->newEvents[] = $event;
    }

    public function apply($event){
        $className = get_class_name($event);
        $methodName = apply.$className;
        if( method_exists($this,$methodName){
             $this->$methodName($event);
        }
    }

    public function applyPriorityChanged(PriorityChanged $event){
        $this->priority = $event->getPriority();
    }

}

Eventsourcing: changing the state

class Issue
{
    ...

    private $newEvents=[];


    public function changePriority($newPriority)
    {
        if($newPriority !== $this->priority){
            $this->record(new PriorityChanged($this->id,$newPriority);
        }
    }

    private function record($event)
    {
        $this->apply($event);
        $this->newEvents[] = $event;
    }

    public function apply($event)
    {
        if( get_class_name($event) === PriorityChanged::class){
             $this->priority = $event->getPriority();
        }
        ..
    }
}

Using single apply method


$issue = new Issue();

$data = $db->get('SELECT * FROM issue WHERE id = 123');

foreach($data as $property => $value){
    $propertySetterName = set.$property;
    $issue->$propertySetterName($value);
}

Load Issue object: CRUD


$issue->changePriority('URGENT');

$db->update('UPDATE ...);

Save Issue object: CRUD

IssueCreated(123, 'Something', 'text', BUG, NORMAL, '2016..')

IssueAssigned(123, TeamManager, '2016-06-12 10:00:00')

PriorityChanged(123,URGENT, '2016-08-11 15:46:33')

IssueDellocated(123,'2016...')

IssueClosed(123,'2016...')

IssueReopened(123, '2018-09-13 11:01:35')

TitleChanged(123, 'Register button doesnt work in IE', '2016...')

id	class_name	object_id	event
...
12	Issue	123	serialized data
13	Issue	123	serialized data
14	User	123	serialized data
15	Issue	92	serialized data
16	User	11	serialized data
17	Issue	323	serialized data
18	Issue	21	serialized data
...

Event store table


$issue = new Issue();

$data = $db->get('SELECT * FROM event_store where object_id=123 and class_name=issue');

foreach($data as $item){
    $event = deserialize($item['event']);
    $issue->apply($event);
}

Load Issue object: EventSourcing


$issue->changePriority('URGENT');

foreach( $issue->getNewEvents() as $newEvent)
{
    $db->insert('INSERT INTO...');
}

Save Issue object: EventSourcing

Any questions on this part?

Hm, how do we show all open issues on our homepage?

We run events for each object and check state?

Hm, how do we show all open issues on our homepage?

We run events for each object and check state?

Any experience with text search engines?

$blogPost = new BlogPost();
$blogPost->setTitle(..)
...

$orm->save($blogPost);


$elasticSearch->save($id,$title,$text,...);

Saving a blog post


$results = $elasticSearch
           ->searchFor('some words');

Searching blog posts for 'some words'

CQRS

Command Query Responsibility Segregation

separate models to write/read
customized data for our need

IssueCreated(123, 'Something', 'text', BUG, NORMAL, '2016..')

~~IssueAssigned(123, TeamManager, '2016-06-12 10:00:00')~~

~~PriorityChanged(123,URGENT, '2016-08-11 15:46:33')~~

~~IssueDellocated(123,'2016...')~~

IssueClosed(123,'2016...')

IssueReopened(123, '2018-09-13 11:01:35')

TitleChanged(123, 'Register button doesnt work in IE', '2016...')

List of open issues: id and title

class OpenIssue
{
    private $id;
    private $title;

    
     public function setId($title)
     {
          $this->id=$id;
     }

     public function getId()
     {
          return $this->id;
     }

    
     public function setTitle($title)
     {
          $this->title=$title;
     }

     public function getTitle()
     {
          return $this->title;
     }
}

OpenIssue: CRUD

class IssueCreatedSubscriber
{
     public function doWork(IssueCreated $event)
     {
          $openIssue = new OpenIssue();
          $openIssue->setid($event->getId());
          $openIssue->setTitle($event->getTitle());
          $orm->save($openIssue);
     }
}

class TitleChangedSubscriber
{
     public function doWork(TitleChanged $event)
     {
          $openIssue = $orm->load($event->getId());
          $openIssue->setTitle($event->getTitle());
          $orm->save($openIssue);
     }
}

class IssueClosedSubscriber
{
     public function doWork(IssueClosed $event)
     {
          $openIssue = $orm->load($event->getId());
          $orm->delete($openIssue);
     }
}

List of open issues: id and title