Building More Resilient Applications

Karim Alibhai | npm install karimsa

Delivering the art & science of retail execution

Pst! We're hiring @

All images & narratives in this talk are purely for satirical purposes. The sequence of events is not in chronological order.

User failure != system failure

The CAP Theorem

  • Consistency

  • Availability

  • Partition tolerance


* Required for distributed systems.

HTTP blah




Web Browser

Unavoidable network partition

Why frontends devs should care

blah blah

Any database solution at all


Unavoidable network partition

Why backends devs should care

Eventual Consistency

Stop building dominoes.

The Broker Pattern

Basic Definition

  • Decouples the work that needs to be completed from the component that wants the work to be completed.
  • Instead of doing the work, the system informs the broker that the work must be done.
  • The broker's core responsibility is orchestration.
  • It is less of "Give me my coffee now" and more "deliver my coffee to me"






*Should be unidirectional.


  • Producer will push work into a queue
  • Consumer will pop work off a queue
  • Important: Pushing / popping are distinct mechanisms -> work is idle for some time
  • Prioritize tasks

    • Priority based on limited resources

    • Priority based on time of day

  • Cancel tasks

  • Dedup tasks

Frontend Example

  • GET /me (1)
  • GET /notifs/count (3)
  • GET /me (1)
  • GET /me/groups (4)
  • GET /me (1)
  • GET /posts/top (2)
  • GET /me (1)
  • GET /posts/top (2)
  • GET /notifs/count (3)
  • GET /me/groups (4)

Backend Example

Without a broker

  • If the database is slow, liking will be slow.
  • If the database is down, liking will be error out the app.

With a broker

  • Try persisting like, if fails (or is slow), cancel and try later.
  • Tell API consumer that it was liked.
  • Store like locally on the browser.

Famous Queue Brokers

  • Frontend: Redux.
  • Backend: Kue, BeeQueue, SQS, etc.

Thanks for listening! | npm install karimsa

Building More Resilient Applications (Part 2)

By Karim Alibhai

Building More Resilient Applications (Part 2)

A story about when all hell breaks loose & lessons learned from it.

  • 1,135