Yaron Minsky / Jane Street Group
Ledgers, Machines, and Markets
What is a ledger?
A sequence of transactions with a well-defined order determining the state of a system.
Where are they used?
- Cryptocurrencies and DeFi (Bitcoin, Ethereum)
- Configuration management (Chubby, Zookeeper)
- Databases (Postgres)
- Exchanges (NASDAQ, CHI-X, BATS)
What's an exchange?
- Durability
- Scale
- Fairness
- Availability
A rendezvous point for buyers and sellers to meet
Requirements:
Simplified exchange
- A single active matching engine/sequencer
- Distributing data via IP multicast
- Bridges consume and propose txns
- Reactors offload work from ME
- Retransmitters prevent message loss
- Passive matching engine for failover
ME
RT
RT
OE
Port
MD
TR
OE
Port
OE
Port
Drop
Port
Cancel
fairy
Auction
fairy
PME
Need for speed
- High peak transaction rates
- Everyone gets every message
- One-in-flight ⇒ Latency determines throughput
- Recovery via replay ⇒ Processing time determines recovery time
Concord
- For bilateral trading relationships
- Highly available and reliable
- Lots of complex trade-flows, many instances
- Large scaling requirements
What went well?
- Reliability
- Scalability
- Composability
- Testability
- Debuggability
What's not to love?
- Specialized to the application
- A single transaction language
- A pain to set up
- Poor performance isolation
- Daily sessions
Aria
- Application agnostic
- Suitable for multiple independent teams
- Good performance isolation
- Shared infra and admin
A general-purpose, shareable Concord
Aria Architecture
- Minimal logic in sequencer
- Proxy layer for injection and (filtered) republishing
Seq
RT
RT
PSeq
Inj
Inj
Repub
Repub
Client
Client
Client
Key features
- Hierarchical topic-space
- Filtering
- Delegation of management
- Consistent ordering
- Test-friendly abstractions
- Snapshots for fast recovery
- Atomic operations
How's it going?
- In production for multiple teams
- Diverse set of use-cases
- Topic-sharing within but not between teams
- Inversion-of-control considered painful
- Still a lot to do!
Takeaways
- Avoid fuzzy-headed "scalability"
- A single single point of failure isn't so bad
- IP Multicast is a neglected gem!
- Performance is about more than performance
Sound like fun?
http://janestreet.com/apply
Ledgers, Machines, and Markets
By Yaron Minsky
Ledgers, Machines, and Markets
- 400
