The New MongoDB Rust Driver

August 10, 2015

 

Kevin Yeh && Sam Rossi

"Why are we exploring a

native Rust driver?"

¯\_(ツ)_/¯

What is Rust?

"Rust is a systems programming language that runs blazingly fast, prevents nearly all segfaults, and guarantees thread safety."

  • zero-cost abstractions
  • move semantics
  • guaranteed memory safety
  • threads without data races
  • trait-based generics
  • pattern matching
  • type inference
  • minimal runtime
  • efficient C bindings

Wow!

Foreign Function Interfaces

  • Zero-cost abstractions between Rust and other languages.

  • Communicates with C, Python, Ruby, Javascript, and many other languages without overhead.

  • Provides C-level performance improvements while leveraging Rust-specific safety guarantees.

How to use the driver

Example: Baseball Player Database

How would you query all the players on a certain team?

{
    "_id" : ObjectId("55a02f52648dca06dce7e5d0"),
    "first_name" : "Jose",
    "last_name" : "Alvarez",
    "bats" : "L",
    "throws" : "L",
    "team" : "LAA",
    "position" : "P",
    "avg" : null,
    "tags" : [ ]
}

Sample

Document

Basic Steps to Query Data

  1. Select the database and collection                         
let db = client.db("mlb");
let coll = db.collection("players");

let filter = Some(doc! { "team" => team });

let mut options = FindOptions::new();
options.projection = Some(doc! {
    "_id" => 0,
    "first_name" => 1,
    "last_name" => 1,
    "position" => 1
});

match coll.find(filter, Some(options)) {
    Ok(cursor) => Ok(cursor),
    Err(e) => err_as_string!(e),
}

 

 

Note: The code segment:

 

 

produces the BSON equivalent to the JSON object:

 

 

(more on that later)

doc! {
    "team" => "BOS"
}

            Step 1

{ "team": "BOS" }
  1. Select the database and collection
  2. Set the query options
    1. "filter" → which documents to select
    2. "projection" (optional) → which fields from the document to return

Basic Steps to Query Data

let db = client.db("mlb");
let coll = db.collection("players");

let filter = Some(doc! { "team" => team });

let mut options = FindOptions::new();
options.projection = Some(doc! {
    "_id" => 0,
    "first_name" => 1,
    "last_name" => 1,
    "position" => 1
});

match coll.find(filter, Some(options)) {
    Ok(cursor) => Ok(cursor),
    Err(e) => err_as_string!(e),
}

            Step 1

      Step 2.2

      Step 2.1

  1. Select the database and collection
  2. Set the query options
    1. "filter" → which documents to select
    2. "projection" (optional) → which fields from the document to return
  3. Check the result for a success

Basic Steps to Query Data

let db = client.db("mlb");
let coll = db.collection("players");

let filter = Some(doc! { "team" => team });

let mut options = FindOptions::new();
options.projection = Some(doc! {
    "_id" => 0,
    "first_name" => 1,
    "last_name" => 1,
    "position" => 1
});

match coll.find(filter, Some(options)) {
    Ok(cursor) => Ok(cursor),
    Err(e) => err_as_string!(e),
}

            Step 1

      Step 2.2

      Step 2.1

      Step 3

  1. Select the database and collection
  2. Set the query options
    1. "filter" → which documents to select
    2. "projection" (optional) → which fields from the document to return
  3. Check the result for a success
  4. Use the results as needed

Basic Steps to Query Data

let mut string = "{\"result\":[".to_owned();

for (i, doc_result) in cursor.enumerate() {
    match json_string_from_doc_result(doc_result) {
        Ok(json_string) => {
            let new_string = if i == 0 {
                json_string
            } else {
                format!(",{}"), json_string)
            };

            string.push_str(&new_string);
        },
        Err(e) => return e,
    }
}

string.push_str("]}");

`Cursor`

implements

`Iterator`

For Instance:

The Three Core Tenets of Rust

  • Safety through the ownership system

  • Concurrency through core structs and features

  • Usability through standard traits and recursive macros 

  • ...and speed!

The Three Core Tenets of Rust

  • Safety through the ownership system

  • Concurrency through core structs and features

  • Usability through standard traits and recursive macros 

Struct Referencing: Looks Good?

Ownership

Rust guarantees speed and safety at runtime by enforcing ownership and lifetimes at compile time.

Since the Client object is borrowed, its lifetime must predictably last at least as long as the database that contains it. ('a)

Arc

Atomic Reference Count is your friend.

How do we pass an Arc of self?

How do we pass an Arc of self?

Types and Traits.

The Three Core Tenets of Rust

  • Safety through the ownership system

  • Concurrency through core structs and features

  • Usability through standard traits and recursive macros 

Mutability in Rust

By default, all variables in Rust are immutable.

 

 

 

 

However, mutability is a bit different in Rust...

 

Interior and Exterior Mutability

By default, all structs and variables in Rust have exterior immutability.

 

Immutable structures can still hold mutable components, as long as they follow the ownership system of Rust:

 

You may have one or the other of these two kinds of borrows, but not both at the same time:

  • one or more references (&T) to a resource.

  • exactly one mutable reference (&mut T).

 

Mutexes and RwLocks

One approach to guaranteeing these rules at compile time is to use RAII locks.

Connection Pools

How we learned to handle mutability,

the hard way.

Traditional Capped Pool

Fixed-length arrays of locks and sockets.

Acquire socket lock, Use socket, Release socket lock.

Hyper

Explicit extraction of lock-free sockets!

Lock the pool

Pop a stream (S)

Return the stream

with a pool reference

Our Friend, the Condvar

Master pool lock with variable-length, lock-free sockets.

If no sockets are available and we are capped at the number of open sockets,  wait on the condition variable until we've been repopulated.

Thank you, Condvar.

The Three Core Tenets of Rust

  • Safety through the ownership system

  • Concurrency through core structs and features

  • Usability through standard traits and recursive macros 

Recursive Macros

The Dark Ages

Not very readable!

let mut update = bson::Document::new();
let mut set = bson::Document::new();

set.insert("director".to_owned(), Bson::String("Robert Zemeckis".to_owned()));
update.insert("$set".to_owned(), Bson::Document(set));

First try

Do we really need a second macro?

let update = doc! {
    "$set" => nested_doc! {
        "director" => Bson::String("Robert Zemeckis".to_owned())
    }
};
#[macro_export]
macro_rules! doc {
    ( $( $k:expr => $v: expr),* ) => {
        {
            let mut doc = Document::new();
            $(
                doc.insert($k.to_owned(), $v);
            )*
            doc
        }
    };
}

#[macro_export]
macro_rules! nested_doc {
    ( $( $k:expr => $v: expr),* ) => {
        Bson::Document(doc!(
            $( $k => $v),*
        ))
    }
}

Not quite there yet...

Explicit types still needed

let update = doc! {
    "$set" => {
        "director" => Bson::String("Robert Zemeckis".to_owned())
    }
};
#[macro_export]
macro_rules! add_to_doc {
    ($doc:expr, $key:expr => ($val:expr)) => {{
        $doc.insert($key.to_owned(), $val);
    }};

    ($doc:expr, $key:expr => [$($val:expr),*]) => {{
        let vec = vec![$($val),*];
        $doc.insert($key.to_owned(), Bson::Array(vec));
    }};

    ($doc:expr, $key:expr => { $($k:expr => $v:tt),* }) => {{
        $doc.insert($key.to_owned(), Bson::Document(doc! {
            $(
                $k => $v
            ),*
        }));
    }};
}

#[macro_export]
macro_rules! doc {
    ( $($key:expr => $val:tt),* ) => {{
        let mut document = Document::new();

        $(
            add_to_doc!(document, $key => $val);
        )*

        document
    }};
}

Ahh...much better!

let doc1 = doc! { "tags" => ["a", "b", "c"] };
let doc2 = doc! { "tags" => ["a", "b", "d"] };
let doc3 = doc! { "tags" => ["d", "e", "f"] };

coll.insert_many(vec![doc1.clone(), doc2.clone(), doc3.clone()], false, None)
    .ok().expect("Failed to execute insert_many command.");

// Build aggregation pipeline to unwind tag arrays and group distinct tags
let project = doc! { "$project" => { "tags" => 1 } };
let unwind = doc! { "$unwind" => ("$tags") };
let group = doc! { "$group" => { "_id" => "$tags" } };
#[macro_export]
macro_rules! bson {
    ([$($val:tt),*]) => {{
        let mut array = Vec::new();

        $(
            array.push(bson!($val));
        )*

        $crate::Bson::Array(array)
    }};

    ([$val:expr]) => {{
        $crate::Bson::Array(vec!(::std::convert::From::from($val)))
    }};

    ({ $($k:expr => $v:tt),* }) => {{
        $crate::Bson::Document(doc! {
            $(
                $k => $v
            ),*
        })
    }};

    ($val:expr) => {{
        ::std::convert::From::from($val)
    }};
}

#[macro_export]
macro_rules! doc {
    () => {{ $crate::Document::new() }};

    ( $($key:expr => $val:tt),* ) => {{
        let mut document = $crate::Document::new();

        $(
            document.insert($key.to_owned(), bson!($val));
        )*

        document
    }};
}

Auto-converted types!

and a little bit about traits...

High focus for Rust 1.0: Extendability

  • Get core features down, provide traits for the community.
  • Large portion of core rust features rely on generic traits.
  • Flexible functional-style programming in a largely procedural language.  

a few useful traits...

  • Conversion Traits: Auto-convert types.

  • Encoder Traits: Auto-encode structs.

  • Error Traits: Handle errors unobtrusively.

  • Dereference Traits: Utilize the auto-dereferencing system.

  • Iterator Traits: Iterate naturally over custom structs.

  • I/O Traits: Perform I/O ops over custom structs.

  • ...and many, many more.

The Three Core Tenets of Rust

  • Safety through the ownership system

  • Concurrency through core structs and features

  • Usability through standard traits and recursive macros 

The Three Core Tenets of Rust

...and speed!

Other Rust language features:

Pattern Matching/Enum types

Suppose you had the following Python function...

def sum_of_first_three_elements(num_list):
    if len(num_list) < 3:
        return None

    return sum(num_list[:3])

How would you implement that in Rust?

 

The second part is easy...

# Python
sum(num_list[:3])
// Rust
num_list[0] + num_list[1] + num_list[2]

But what would the return type be?

There's no "null" in Rust!

Solution: Optional types

pub enum Option<T> {
    None,
    Some(T),
}
fn sum_of_first_three_elements(num_list: &[i32]) -> Option<i32> {
    if num_list.len() < 3 {
        None
    } else {
        Some(num_list[0] + num_list[1] + num_list[2])
    }
}

Example function implemented in Rust

It might return an int...but it might not

"variants": almost like constructors

"enumerated type"

note: "T" is the generic type of the parameter

Obvious question: But how do you use options?

Not-so obvious answer (unless you've done functional programming before): pattern matching!

Pattern matching

Like switch statements...but actually useful!

General form

match <expression> {
   <first pattern> => <resulting expression>,
   <second pattern> => <resulting expression>,
   ...
   // Optional catch-all
   _ => <resulting expression>
}

Example: pattern matching an option

match maybe_an_int {
    Some(i) => println!("The integer is: {}", i),
    None => println!("There is no integer :(")
}

"match" is actually an expression

fn say_hey(name: Option<&str>) {
   let person = match name {
       Some(string) => string,
       None => "you"
   };

   println!("Hey, {}!", person);
}

say_hey(Some("Joe"); // prints "Hey, Joe!"
say_hey(None);       // prints "Hey, you!"

Advanced pattern matching

pub enum Bson {
    FloatingPoint(f64),
    I32(i32),
    I64(i64),
    Binary(BinarySubtype, Vec<u8>),
    // other types omitted
}

match bson_document.get("some_key") {
   // Explicit fallthrough, and ignored capture of single variable
   Some(&Bson::I32(_)) |
   Some(&Bson::I64(_)) => println("I got an int!"),

   // Multi-line results
   Some(&Bson::FloatingPoint(f)) => {
      let i = f as i32;
      println!("I didn't get an int...but if I did, it would look like this: {}", f);
   }

   // Ignored capture of multiple variables
   Some(&Bson::Binary { .. }) => println!("I got some bits"),

   // "wildcard" matches everything else
   _ => println!("I got a rock...")
}

Benefits of pattern matching

  • Flexible
  • Completely safe
    • If you miss a variant (without specifying a wild card), the compiler will tell you
  • Really freaking fast

Bonus time

You can implement methods on enums just like other structs!

impl Bson {
    fn print_value(&self) {
        match self {
            Bson::I32(i) => println!("{}", i),
            Bson::I64(i) => println!("{}", i),
            Bson::FloatingPoint(f) => println!("{}", f),
            ...
        }
    }
}

let bson_int = Bson::I32(17);
bson_int.print_value(); // prints "17"

Driver Status

Complete 

  • bson library

    • OID generation

    • stable macro

  • connection strings
  • wire protocol

  • CRUD, commands, and cursors

  • bulk writes

  • connection pooling

  • GridFS

  • server discovery and monitoring (SDAM)

  • command monitoring (APM)

  • SCRAM-SHA-1 auth

  • server selection & failover

Up Next

  • shard tagging and other server commands

  • SSL Support

  • Other auth mechanisms

  • __

Learn more...

...and join us!

Questions?

The New MongoDB Rust Driver

By Kevin Yeh

The New MongoDB Rust Driver

  • 148