Yaron Minsky / Jane Street Group

Ledgers, Machines, and Markets

What is a ledger?

A sequence of transactions with a well-defined order determining the state of a system.

Where are they used?

  • Cryptocurrencies and DeFi (Bitcoin, Ethereum)
  • Configuration management (Chubby, Zookeeper)
  • Databases (Postgres)
  • Exchanges (NASDAQ, CHI-X, BATS)

What's an exchange?

  • Durability
  • Scale
  • Fairness
  • Availability

A rendezvous point for buyers and sellers to meet

Requirements:

Simplified exchange

  • A single active matching engine/sequencer
  • Distributing data via IP multicast
  • Bridges consume and propose txns
  • Reactors offload work from ME
  • Retransmitters prevent message loss
  • Passive matching engine for failover

ME

RT

RT

OE

Port

MD

TR

OE

Port

OE

Port

Drop

Port

Cancel

fairy

Auction

fairy

PME

Need for speed

  • High peak transaction rates
  • Everyone gets every message
  • One-in-flight ⇒ Latency determines throughput
  • Recovery via replay ⇒ Processing time determines recovery time

Concord

  • For bilateral trading relationships
  • Highly available and reliable
  • Lots of complex trade-flows, many instances
  • Large scaling requirements

What went well?

  • Reliability
  • Scalability
  • Composability
  • Testability
  • Debuggability

What's not to love?

  • Specialized to the application
  • A single transaction language
  • A pain to set up
  • Poor performance isolation
  • Daily sessions

Aria

  • Application agnostic
  • Suitable for multiple independent teams
  • Good performance isolation
  • Shared infra and admin

A general-purpose, shareable Concord

Aria Architecture

  • Minimal logic in sequencer
  • Proxy layer for injection and (filtered) republishing

Seq

RT

RT

PSeq

Inj

Inj

Repub

Repub

Client

Client

Client

Key features

  • Hierarchical topic-space
    • Filtering
    • Delegation of management
  • ​Consistent ordering
  • Test-friendly abstractions
  • Snapshots for fast recovery
  • Atomic operations

How's it going?

  • In production for multiple teams
  • Diverse set of use-cases
  • Topic-sharing within but not between teams
  • Inversion-of-control considered painful
  • Still a lot to do!

Takeaways

  • Avoid fuzzy-headed "scalability"
  • A single single point of failure isn't so bad
  • IP Multicast is a neglected gem!
  • Performance is about more than performance

Sound like fun?

http://janestreet.com/apply