Implementing CQRS with Akka

Michał Tomański

@michaltomanski

CQRS

What is it about?

  • Command Query Responsibility Segregation
  • Design/architecture pattern
class UserService {
  createUser(User user): Unit
  getUser(long id): User
}
class UserWriteService {
  createUser(User user): Unit
}

class UserReadService {
  getUser(long id): User
}

And that's basically it

Let's mix it!

Event Sourcing

  • The state of an object is equivalent to all past events applied one by one
  • Persist all the events to a journal
  • Events are immutable, they are facts
  • Events can be replayed to restore the state

Event Sourcing

  • System receives command
  • Command gets validated
  • Command generates events
  • Events get persisted
  • Events are applied on the state

DeactivateUser(id = 1)

Is the user with id=1 active?

UserDeactivated(id = 1)

UserDeactivated(id = 1) goes to DB

isActive = false

Why?

  • Persist facts
  • Write only, never modify/delete
  • Better domain modelling
  • Temporal queries
  • Free audit logs / BI data

- Cool, but how do we query this thing? 

- Well, you don't

Why bother?

  • Everything said earlier
  • Suitable database engines for each side
  • Independent read and write side scalability
  • Independent data replication factor
  • Read or write oriented domain
  • Better performance with prepared views
  • Loose coupling

Let's build something!

Awesome Timer

®

  • Measures the time
  • Online
  • Users see other users' results (rivalry)

Overview

Awesome Timer

®

Write side

  • Accept user's new time
  • Remove the user's time (in case of unintentionally stopping the timer)
  • Add +2s (penalty)

Rare

Awesome Timer

®

Read side

Show to the user his:

  • Best single time
  • Best average of 5
  • Best average of 12
  • Best average of 100
  • Current average of 5, 12, 100
  • Last 100 times

Show to the user other people's:

  • Best single time
  • Best average of 5
  • Best average of 12
  • Best average of 100
  • Current average of 5, 12, 100

Awesome Timer

®

Read side

Average of 5: 16, (20), 18, 18, (15) 

= (16 + 18 + 18) /3

Awesome Timer

®

Writes and reads

  • Extremely simple write side (just persist the username and the time), but with (hopefully) a lot of traffic
  • Quite complicated and expensive reads (calculating best average of 5 among thousands of entries, querying multiple users)
  • Typical CRUD is not the case here

- Ok, how do we do that? We build all of those persistence, projections etc?

  • A toolkit focused on actors model
  • Useful for concurrent, distributed and message driven systems
  • High-level abstraction

Actor

  • Encapsulate state and behavior
  • React to a message
  • Supervision

Actor

class Speedcuber extends Actor { 
  var times = Nil

  def receive = {
    case AddTime(time: Long) => times :+ time
  }
}

actor ! AddTime(500)

Persistent Actor

  • Persist every valid event to the journal
  • Do your business logic after persistence
  • Replay all the events when needed

Persistent Actor

class Speedcuber extends PersistentActor {
  var times = Nil

  override def receiveCommand: Receive = {
    case addTime: AddTime =>
      persist(addTime) { event =>
        times = times :+ event.time
      }
  }  
  
  override def receiveRecover: Receive = {
    case addTime: AddTime => times = times :+ event.time
    case RecoveryCompleted => println("Events recovery completed")
  }

  override def persistenceId: String = "speedcuber"
}

Persistent Actor

As a domain entity

Persistent Actor

Journal (Cassandra)

Persistent Actor

Storage plugins

  • Cassandra, DynamoDB, Mongo, SQL etc.
  • Storage initialization done automagically by the plugin, lazily or eagerly
  • Specified in .conf file
akka.persistence.journal.plugin = "cassandra-journal"
akka.persistence.snapshot-store.plugin = "cassandra-snapshot-store"

Projection

Akka Persistence Query as a projection

Akka persistence query

Postgresql

Projection

Akka Persistence Query as a projection

  • Get source of events by tag from journal
  • Starting from last offset
  • Map the stream with async function
  • At the end, save the offset of the event
  • When handling the event, construct the view and upsert to the database
val s = readJournal.eventsByTag("Speedcuber")

val s = readJournal.eventsByTag("Speedcuber", state.offset)
s.mapAsync(1)(handleEvent).runWith(Sink.foreach(offset => self ! SaveOffset(offset)))

def handleEvent = {
  case EventEnvelope(offset, _, _, event: BestAvgChanged) =>
    viewBuilder.upsertBestAvgView(BestAvg(event.user, event.millis)).map(_ => offset)
val s = readJournal.eventsByTag("Speedcuber", state.offset)

val s = readJournal.eventsByTag("Speedcuber", state.offset)
s.mapAsync(1)(handleEvent)
val s = readJournal.eventsByTag("Speedcuber", state.offset)
s.mapAsync(1)(handleEvent).runWith(Sink.foreach(offset => self ! SaveOffset(offset)))

Cluster Singleton

Projection as a Cluster Singleton

The oldest node

Controllers

Communicating with the world

Eventual consistency

Should I care?

  • Read data will be consistent with what is inside the event store... eventually.
  • Getting an updated view by the user might take a while
  • But isn't it always the case?
  • Data is not wrong, it's just old

Cubing time

Live demo

Summary

We have a service...

  • That can have read and write model scaled independently
  • With databases suitable for read and write
  • Easy to integrate with other services via events messaging
  • Easy to be scaled horizontally with state recovered
  • With complete and immutable events history
  • With prepared expensive views
  • Easy to be tested and debugged (just dump events from prod)

Implementing CQRS with Akka

Michał Tomański

@michaltomanski

slides.com/michaltomanski/cqrs

github.com/michaltomanski/cqrs-demo

vote.scalaua.com

Challenges

Eventual consistency

Should I care?

  • Well, there might be a problem while making domain validations against views...
  • When? When not?
  • Send a compensation event
  • Tweak with polling times
  • Keep track of inconsistency issues
  • ... Or do not design your (micro)service this way

¯\_(ツ)_/¯

Cluster sharding

As a domain entities sharding

Node 1

Node 2

Events evolution

  • We want an actor and the projection to always work on the newest events versions
  • Events stored in journal can be upcasted to the newest version during replay by adapters
  • Good serialization method may also help

http://doc.akka.io/docs/akka/current/scala/persistence-schema-evolution.html

Entities startup

  • Use rememberEntities flag
  • Rebalancing shards to another nodes
  • Startup of the application
  • Journal might get overwhelmed 
  • Use snapshotting
  • Passivation after idle timeout

Serialization

  • Domain events persistence
  • Internal events persistence (e.g. shard coordinator)
  • Remoting for domain events
  • Remoting for internal events

Remoting

  • Akka is designed for remoting
  • Just a matter of configuration

Persistent Actor

Event adapters

  • Tagging - e.g. fetching only domain events
  • Events versioning
event-adapters = {
  speedcuber = "com.mtomanski.timer.infrastructure
    .akka.adapter.SpeedcuberEventsTaggingAdapter"
}

event-adapter-bindings = {
  "com.mtomanski.timer.domain.model.Speedcuber" = [speedcuber]
}
class SpeedcuberEventsTaggingAdapter extends WriteEventAdapter {
  override def toJournal(event: Any): Any = Tagged(event, Set("Speedcuber"))
  ... 
}
Made with Slides.com