Building real-time transport monitoring

with Event Sourcing and Akka Persistence

James Dam, Software Engineer

Pavel Kudinov, Engineering Manager

RedMart Delivery

RedMart Delivery

Real-Time + History

Why not CRUD

UPDATE replaces previous values

Need extra effort to create changelog

Need extra effort to be event-driven

Actor model

Lightweight

No shared state

Communication via messages

Each driver, order, cart is an actor!

Why Akka?

Performance

Scalability

Fault tolerance

Event sourcing

class DriverActor extends Actor {

  var state = DriverActorState(location = None)

  override def receive = {

    case UpdateLocationCommand(location) =>
      state = DriverActorState(Some(location))

    case GetStateCommand =>
      state.location match {
        case None => sender() ! Failure(new UninitializedException())
        case _    => sender() ! Success(state)
      }
  }
}

Akka persistence

Store events and recover state of the actor

class DriverActor(driverId: String) extends PersistentActor {

  override val persistenceId = "driver-" + driverId

  ....

  override def receiveCommand = {
    case UpdateLocationCommand(location) =>
      persist(LocationUpdatedEvent(location))(eventPersisted)
  }

  def eventPersisted(event: Event) = {
    updateState(event)
  }

  override def receiveRecover = {
    case event: Event => 
      updateState(event) // Replay events from Journal
  }
}

class DriverActor(driverId: String) extends PersistentActor {

  ......

  def eventPersisted(event: Event) = {
    updateState(event)
    if ( /* need to save snapshot */ ) {
      saveSnapshot(state)
    }
  }

  override def receiveRecover = {
    case SnapshotOffer(_, snap: DriverActorState) =>
      state = snap
    case event: Event =>
      updateState(event)
  }

}

Snapshots

Event Subscription

class DriverActor(driverId: String) extends PersistentActor {

  val mediator = DistributedPubSub(context.system).mediator

  def eventPersisted(event: Event) = {
    ....
    mediator ! Publish("driver-" + driverId, event)
    ....
  }
}


class DriverEventListenerActor(driverId: String) extends Actor {

  val mediator = DistributedPubSub(context.system).mediator
  mediator ! Subscribe("driver-" + driverId, self)

  def receive = {
    case e: Event => 
        // send to websocket
        // do other processing if required
  }
}

How to query all drivers?

CQRS & Akka persistence

Stream Events from Journal

1. Read events from journal as they appear
2. Transform DriverView using new journal entry
3. Execute (buffered) INSERT into the Query Side
4. Save the journal offset

class DriverUpdatedEventProcessor extends Actor {
    
    val readQuery = PersistenceQuery(context.system)
        .readJournalFor[CassandraReadJournal](CassandraReadJournal.Identifier)
    
    def startProcessing() = {
        readQuery
          .eventsByTag("DriverUpdated", this.savedOffset)
          .groupedWithin(100, 10.seconds)
          .mapAsync(1) { envelopes =>
            processEvents(envelopes.map(_.event)).map(_ => envelopes.last.offset);
          }
          .map { offset => self ! SaveOffsetCommand(offset) }
          .runWith(Sink.ignore)
    }

    
    def processEvents(events: Seq[Any]): Future[Unit] = {
        /* Write data to Amazon Redshift or other DB */
    }
}

How to scale?

Akka cluster

Coordinator

Shard Region 1

Shard A

Actor 1

Actor 2

Shard B

Shard Region 2

Shard E

Actor 3

Shard Region 3

Shard D

Actor 4

Actor 5

Actor 6

Node 1

Node 2

Node 3

Q&A

Interested in joining RedMart?

Solve a coding challenge at geeks.redmart.com/tag/puzzles