Rust APIs for performance & type safety

a case study

By Filippo Costa

https://twitter.com/neysofu

https://github.com/neysofu

1. Why do rustaceans care? 🦀

2. FIX

3. Some ideas for fix-toy

4. Looking back 🤔

Doing some archeology

What is FIX?

Financial Information eXchange protocol

  1. A domain-specific data transfer protocol.
  2. Popularized in the '90s.
  3. Text-based, human-readable. Mostly.

  4. Format: tag=value pairs, separated by SOH character.

https://www.fixtrading.org/standards/tagvalue-online/

8=FIX.4.4
9=148
35=D
34=1080
49=TESTBUY1
52=20180920-18:14:19.508
56=TESTSELL
111=636730640278898634
15=USD
21=2
38=7000
40=1
54=1
55=MSFT
60=20180920-18:14:19.492
10=092

https://www.fixtrading.org/standards/tagvalue-online/

https://www.onixs.biz/fix-dictionary/4.4/fields_by_name.html

TW: FIX is atrocious.

https://en.wikipedia.org/wiki/Central_limit_theorem

https://en.wikipedia.org/wiki/Central_limit_theorem

https://www.fixtrading.org/standards/fix-session-layer-online/

What we want out of fix-toy

A crate for FIX newcomers as well as demanding users.

1. Not standard-compliant: data and NumInGroup.

2. No streaming support!

3. DevX considerations.

4. N+1 allocations.

5. No choice of BuildHasher.

Introducing the FIX dictionary

A XML specification for FIX messages

https://github.com/quickfix/quickfix/

The good...

  • Type-safe
  • Flexible querying
  • Easily clonable

 

...and the ugly

  • Expensive initialization
  • Queries are expensive
  • Huge memory consumption

Adding a dependency to the dictionary

  • Parsing requires taking a &.
  • DevX suggests Arc<Dictionary>. 💡

Woeful restrictions for downstream users. 🐌

streaming support

FIX doesn't integrate any framing mechanism. 🖼️

Tokio  to the rescue! 🚀

datagrams vs streams?

https://crates.io/crates/tokio-util

https://crates.io/crates/bytes

So, we're making the api more complete

But what did we lose?

 

Devs now have to think about dictionaries, Tokio, they can't directly use &[u8]. Every user seems to must care about every abstraction we introduce.

 

Soon enough: openssl & rustls, decimal, and others.

We can solve these problems with traits: the most comprehensive type abstraction tool we have in Rust.

💡

Much more we could do 👩🏼‍‍🔬

Huge dependencies like Dictionary are fertile grounds for trait-based abstractions.

https://docs.rs/diesel/2.0.0/diesel/query_dsl/methods/trait.LoadQuery.html

RFC 1598 - GATs

RFC 3185 - async fn in traits

RFC 2532 - default associated types

feature(generic_const_exprs)

Traits enable late binding implementation details

We often introduce traits out of necessity, reacting to emerging requirements.

What if we were more proactive about this? ⚡

Does this type let me do that?

What is it exactly that I need here?

➡️

Public crate design differs greatly from private codebases.

  1. Traits allow us to be more agnostic towards downstream users' situations and use cases.
  2. This is only possible because of the added structure.

https://0ver.org/

Don't be scared to populate your crates with many traits.

Your SemVer will be alright. Really.

Food for thought

  1. Abstracting Dictionary away lets us explore truly zero-cost codegen solutions. 🏃🏻‍♂️
  2. Placing field getters into a trait would allow code reuse for JSON, Protobuf, and other FIX formats.
  3. Surrogate traits with associated would help to keep APIs slim and simple by introducing hierarchies.

Thank you!

Questions?

Say hi 👋🏻

https://twitter.com/neysofu

filippo@neysofu.me

 

https://slides.com/filippoc/rustlab2022

Made with Slides.com