Intro to EventSourcing & CQRS

my story

RegistrationController

  • validate data
  • create user object
  • persist user object
  • create activation email
  • send email
  • save user to elasticsearch

RegistrationController

RegistrationService

ORM

RegistrationEmailManager

Mailer

EmailComposer

ORM

ElasticSearchManager

UserImportManager

UserCsvProcessor

UserValidationService

UserImportFetcher

ShoppingService

OrderService

UserManager

Product

Category

  • id
  • title
  • description
  • categoryId
  • price
  • vendorId
  • createdAt
  • id
  • title
  • parentId
  • createdAt

Product

Category

  • id
  • title
  • description
  • categoryId
  • category_level_0
  • category_level_1
  • category_level_2
  • price
  • vendorId
  • createdAt
  • id
  • title
  • parentId
  • category_level_0
  • category_level_1
  • category_level_2
  • createdAt

Product should be visible in all parent categories

Show products on site

class ProductsRepository
{

    public function getProducts($categoryId,$categoryLevel,...){
        ...
        $this->addWhere('category_level_'.$categoryLevel,$categoryId);
        ...
    }
}

Edit category: change parent

Pretty fine on 100 products

Timeout on 10.000 products

Product

Category

  • id
  • title
  • description
  • categoryId
  • category_level_0
  • category_level_1
  • category_level_2
  • discount
  • price
  • vendorId
  • createdAt
  • id
  • title
  • parentId
  • category_level_0
  • category_level_1
  • category_level_2
  • createdAt

We are opening special category with discounts

Show product on site

class ProductsRepository
{

    public function getProducts($categoryId,$categoryLevel,...){
        ...

        if( $categoryId === '34242'){
            $this->addWhere('discount is not null');
        }else{
            $this->addWhere('category_level_'.$categoryLevel,$categoryId);
        }
        ...
    }
}

Get product price

class Product
{

    public function getPrice(){
        ...

        if( !is_null($this->discount)){
            $price = $this->price;
            $price *= (100 - $this->discount)/100;
            return $price;

        }
        return $this->price;
    }
}

Miro Svrtan

senior engineer / trainer

@msvrtan

 

miro@mirosvrtan.me

as web is growing

more and more core business systems are web

simple CMS

simple webshops

existing practices and pattern

existing practices and pattern

new ideas

eventsourcing

command query responsibility segregation

Domain

Our problem

processes

framework, DB, servers..

Infrastructure

Application

class RegisterController
{
    public function registerUser($form){
       $....
       $this->validate(...);
       $....
       $this->fillUpUserObject(...);
       $....
       $this->save(...);
       $....
       $this->sendConfirmationEmail(...);
       $this->saveUserInTextSearchEngine(...);
       $....
       $this->showView(...);
    }
}
class RegisterController
{
    public function registerUser($form){
       $....
    }
    public function registerUserApi($apiData){
       $....
    }
}

class CsvImportController
{
    public function importUsersCsv(){
       $data = $this->readFromDisk('somefile.csv');
       $....
    }
}


class RemoteImportController
{
    public function importFromRemoteSystem(){
       $data = curl(..);
       $....
    }
}

something happened

do something else

event

subscriber

UserRegisteredEvent

SendActivationEmail

SaveInTextSearchEngine

class UserRegisteredEvent
{
    private $userId;

    public function __constuctor($userId)
    {
        $this->userId=$userId;
    }

    public function getUserId()
    {
       return $this->userId;
    }
}

class SendActivationEmail
{
    ... 

    public function doWork(UserRegisteredEvent $event){
       $....
    }

    ...
}

class SaveUserInTextSearchEngine
{
    ...

    public function doWork(UserRegisteredEvent $event){
       $....
    }

    ...
}

Message bus

How do we connect event to subcriber?


class MessageBus{

    private $list=[];

    public function add(string $eventName, array $subscriberNames)
    {
        $this->list[$eventName] = $subscriberNames;
    }

    public function doWork($event)
    {
        foreach($this->findSubscribers($event::class) as $subscriber)
        {
             $subscriber->doWork($event);
        }
    }
}
class RegisterController
{
    public function registerUser($form){
       $....
       $this->validate(...);
       $....
       $this->fillUpUserObject(...);
       $....
       $this->save(...);
       $....
       $messagebus->doWork(new UserRegisteredEvent($userId));
       $this->showView(...);
    }

    public function registerApiUser($json){
       $....
       $messagebus->doWork(new UserRegisteredEvent($userId));
       $this->showView(...);
    }
}

Changes

  • MessageBus
  • UserRegisteredEvent
  • SendActivationEmail
  • SaveInTextSearchEngine

Connect events->subscribers

we just registered a user

everything about sending email

everything about text search

How hard is it to modify now?

Switch email provider

  • MessageBus
  • UserRegisteredEvent
  • SendActivationEmail
  • SaveInTextSearchEngine

change this class

Change text search engine

  • MessageBus
  • UserRegisteredEvent
  • SendActivationEmail
  • SaveInTextSearchEngine

change this class

Stop sending activation emails

  • MessageBus
  • UserRegisteredEvent
  • SendActivationEmail
  • SaveInTextSearchEngine

delete this class

Registration

is still duplicated...

tell

do

command

handler

RegisterUserCommand

RegisterUserHandler

class RegisterUserCommand
{
    private $userId;
    private $username;
    private $email;

    public function __constuctor($userId,$username,$email)
    {
        $this->userId    = $userId;
        $this->username = $username;
        $this->email    = $email;
    }

    public function getUserId()
    {
       return $this->userId;
    }

    public function getUsername()
    {
       return $this->username;
    }

    public function getEmail()
    {
       return $this->email;
    }
}
class RegisterUserHandler
{
    ...

    public function handle(RegisterUserCommand $command)
    {
        $...
        $...
        $...
        $messagebus->doWork(new UserRegisteredEvent($userId));
    }

}
class RegisterController
{
    public function registerUser($form){
       $....
       $this->validate(...);
       $....
       $commandbus->handle(new RegisterUserCommand(...));
       $this->showView(...);
    }

    public function registerApiUser($json){
       $....
       $commandbus->handle(new RegisterUserCommand(...));
       $this->showView(...);
    }
}

CHE

command handler event

command pattern

event pattern

RegisterUserCommand

RegisterUserHandler

UserRegisteredEvent

SendActivationEmail

SaveToTextSearchEngine

RegistrationController

ImportController

CLI

AdminController

Commands

  • action to take
    • RegisterUser
    • SendEmail
    • BuyMilk
  • command bus
  • 1 command
    • exactly 1 handler

Events

  • what happened
    • UserRegistered
    • EmailSent
    • MilkBought
  • event bus
  • 1 event 
    • 0,1,2,3,... subscribers

User still waits for email to be sent :(

  • create SendActivationEmailCommand & Handler
  • make SendActivationEmailSubscriber launch command instead of doing work
  • add queueable command bus

Any questions on this part?

id 123456789
title Register button doesnt work in IE
assigned null
type BUG
status OPEN
priority URGENT
createdAt 2016-06-11 22:22:11
updatedAt 2019-01-09 11:01:35
closedAt

Issue tracker: Issue

How long is this urgent bug opened?

No one is assigned to it?

Lets check the logs

IssueOpened: 123456789 [2016-06-11 22:22:11]

IssueClosed: 123456789 [2016-06-12 12:00:11]

  • We are missing details now :(
  • Lets add logging when priority changes
  • 2months later: lets add logging when user is assigned/removed

New company policy: log everything

Until lazy developers forget to log something

We just lost valuable data :(

Logging stuff


class EventBus extends MessageBus
{

    public function doWork($event)
    {
        parent::doWork($event);
        $this->saveSomewhere($event);
    }
}
id event createdAt
1 O:19:"IssueOpened":3:{s:26:"IssueOpenedid";s:15:"123";s:26:"IssueOpenedname";s:5:"Title";..;} 2010-03-17 11:22:33
2 ... ...
... ... ..
... ... ..
... O:17:"IssueTitleChanged":3:{s:26:"IssueTitleChanged";s:3:"124";s:4:"joe2";} 2018-09-13
10:45		 
class UpdateIssueHandler
{
    public function handle(UpdateIssue $command){
       $....
       $logger->info('Title on issue $id change to ' . $title);
    }
}
class UpdateIssueHandler
{
    public function handle(UpdateIssue $command){
       $....
       $eventbus->doWork(new IssueTitleChanged($id,$title));
    }
}

I'm too lazy to add Event here

What if instead of logging, we create object state from events?

IssueCreated(123, 'Something', 'text', BUG, NORMAL, '2016..')

IssueAssigned(123, TeamManager, '2016-06-12 10:00:00')

PriorityChanged(123,URGENT, '2016-08-11 15:46:33')

IssueDellocated(123,'2016...')

IssueClosed(123,'2016...')

IssueReopened(123, '2019-01-09 11:01:35')

TitleChanged(123, 'Register button doesnt work in IE', '2016...')

Issue object is now in same state as in CRUD

Event sourcing

class Issue
{
    private $id;
    private $title;
    private $text;
    private $assigned;
    private $type;
    private $status;
    private $priority;

    public function changePriority($newPriority){
        $this->priority = $newPriority;
    }

}

CRUD

class Issue
{
    ...

    public function changePriority($newPriority){
        if($newPriority !== $this->priority){
            $this->record(new PriorityChanged($this->id,$newPriority);
        }
    }
}

Eventsourcing: change priority

class Issue
{
    ...

    private $newEvents=[];


    public function changePriority($newPriority){
        if($newPriority !== $this->priority){
            $this->record(new PriorityChanged($this->id,$newPriority);
        }
    }

    private function record($event){
        $this->apply($event);
        $this->newEvents[] = $event;
    }

    public function apply($event){
        $className = get_class_name($event);
        $methodName = apply.$className;
        if( method_exists($this,$methodName){
             $this->$methodName($event);
        }
    }
}

Eventsourcing: record & apply

class Issue
{
    ...

    private $newEvents=[];


    public function changePriority($newPriority){
        if($newPriority !== $this->priority){
            $this->record(new PriorityChanged($this->id,$newPriority);
        }
    }

    private function record($event){
        $this->apply($event);
        $this->newEvents[] = $event;
    }

    public function apply($event){
        $className = get_class_name($event);
        $methodName = apply.$className;
        if( method_exists($this,$methodName){
             $this->$methodName($event);
        }
    }

    public function applyPriorityChanged(PriorityChanged $event){
        $this->priority = $event->getPriority();
    }

}

Eventsourcing: changing the state

class Issue
{
    ...

    private $newEvents=[];


    public function changePriority($newPriority)
    {
        if($newPriority !== $this->priority){
            $this->record(new PriorityChanged($this->id,$newPriority);
        }
    }

    private function record($event)
    {
        $this->apply($event);
        $this->newEvents[] = $event;
    }

    public function apply($event)
    {
        if( get_class_name($event) === PriorityChanged::class){
             $this->priority = $event->getPriority();
        }
        ..
    }
}

Using single apply method


$issue = new Issue();

$data = $db->get('SELECT * FROM issue WHERE id = 123');

foreach($data as $property => $value){
    $propertySetterName = set.$property;
    $issue->$propertySetterName($value);
}

Load Issue object: CRUD


$issue->changePriority('URGENT');

$db->update('UPDATE ...);

Save Issue object: CRUD

IssueCreated(123, 'Something', 'text', BUG, NORMAL, '2016..')

IssueAssigned(123, TeamManager, '2016-06-12 10:00:00')

PriorityChanged(123,URGENT, '2016-08-11 15:46:33')

IssueDellocated(123,'2016...')

IssueClosed(123,'2016...')

IssueReopened(123, '2019-01-09 11:01:35')

TitleChanged(123, 'Register button doesnt work in IE', '2016...')

id class_name object_id event
...
12 Issue 123 serialized data
13 Issue 123 serialized data
14 User 123 serialized data
15 Issue 92 serialized data
16 User 11 serialized data
17 Issue 323 serialized data
18 Issue 21 serialized data
...

Event store table


$issue = new Issue();

$data = $db->get('SELECT * FROM event_store where object_id=123 and class_name=issue');

foreach($data as $item){
    $event = deserialize($item['event']);
    $issue->apply($event);
}

Load Issue object: EventSourcing


$issue->changePriority('URGENT');

foreach( $issue->getNewEvents() as $newEvent)
{
    $db->insert('INSERT INTO...');
}

Save Issue object: EventSourcing

Any questions on this part?

Hm, how do we show all open issues on our homepage?

We run events for each object and check state?

Hm, how do we show all open issues on our homepage?

We run events for each object and check state?

Any experience with text search engines?

$blogPost = new BlogPost();
$blogPost->setTitle(..)
...

$orm->save($blogPost);

$elasticSearch->save($id,$title,$text,...);

Saving a blog post


$results = $elasticSearch
           ->searchFor('some words');

Searching blog posts for 'some words'

CQRS

Command Query Responsibility Segregation

  • separate models to write/read
  • customized data for our need

IssueCreated(123, 'Something', 'text', BUG, NORMAL, '2016..')

IssueAssigned(123, TeamManager, '2016-06-12 10:00:00')

PriorityChanged(123,URGENT, '2016-08-11 15:46:33')

IssueDellocated(123,'2016...')

IssueClosed(123,'2016...')

IssueReopened(123, '2019-01-09 11:01:35')

TitleChanged(123, 'Register button doesnt work in IE', '2016...')

List of open issues: id and title

class OpenIssue
{
    private $id;
    private $title;

    
     public function setId($title)
     {
          $this->id=$id;
     }

     public function getId()
     {
          return $this->id;
     }

    
     public function setTitle($title)
     {
          $this->title=$title;
     }

     public function getTitle()
     {
          return $this->title;
     }
}

OpenIssue: CRUD

class IssueCreatedSubscriber
{
     public function doWork(IssueCreated $event)
     {
          $openIssue = new OpenIssue();
          $openIssue->setid($event->getId());
          $openIssue->setTitle($event->getTitle());
          $orm->save($openIssue);
     }
}

class TitleChangedSubscriber
{
     public function doWork(TitleChanged $event)
     {
          $openIssue = $orm->load($event->getId());
          $openIssue->setTitle($event->getTitle());
          $orm->save($openIssue);
     }
}

class IssueClosedSubscriber
{
     public function doWork(IssueClosed $event)
     {
          $openIssue = $orm->load($event->getId());
          $orm->delete($openIssue);
     }
}

List of open issues: id and title

OMG

Option 1: CRUD + LOG

Option 2: ES + CQRS + CRUD

quick

not losing data, scalable

CHE + CQRS example

Admin:Listings to verify

Listings to verify

Listings to verify

v2

ListingCreatedSubscriber

ListingRejectedSubscriber

EventSomethingHappenedSubscriber

...

v2

I didn't cover

  • UUID
  • DDD
    • strategical
    • tactical
  • aggregates
  • sagas/process managers
  • popular frameworks
    • broadway
    • prooph
  • message bus implementations
    • SimpleBus
    • Tactician

Quick recap

  • intention instead of managers/processors/..
  • domain/application/infrastructure separation
  • try to solve next problem
    • without web server, database or framework
  • command /event patterns help decouple code
  • events can be better than logging
  • you can pick some ideas and use them today

Thank you!

@msvrtan

Feedback appreciated: https://joind.in/talk/e8636

Questions?

Intro to ES+CQRS (SunshinePHP 2019)

By Miro Svrtan

Made with Slides.com

Intro to ES+CQRS (SunshinePHP 2019)

Introduction to eventsourcing and command query responsibility segragation

  • 2,358

More from Miro Svrtan