The New MongoDB Rust Driver
August 10, 2015
Kevin Yeh && Sam Rossi
"Why are we exploring a
native Rust driver?"
¯\_(ツ)_/¯
What is Rust?
"Rust is a systems programming language that runs blazingly fast, prevents nearly all segfaults, and guarantees thread safety."
- zero-cost abstractions
- move semantics
- guaranteed memory safety
- threads without data races
- trait-based generics
- pattern matching
- type inference
- minimal runtime
- efficient C bindings
Wow!
Foreign Function Interfaces
-
Zero-cost abstractions between Rust and other languages.
-
Communicates with C, Python, Ruby, Javascript, and many other languages without overhead.
-
Provides C-level performance improvements while leveraging Rust-specific safety guarantees.
How to use the driver
Example: Baseball Player Database
How would you query all the players on a certain team?
{
"_id" : ObjectId("55a02f52648dca06dce7e5d0"),
"first_name" : "Jose",
"last_name" : "Alvarez",
"bats" : "L",
"throws" : "L",
"team" : "LAA",
"position" : "P",
"avg" : null,
"tags" : [ ]
}
Sample
Document
Basic Steps to Query Data
- Select the database and collection
let db = client.db("mlb");
let coll = db.collection("players");
let filter = Some(doc! { "team" => team });
let mut options = FindOptions::new();
options.projection = Some(doc! {
"_id" => 0,
"first_name" => 1,
"last_name" => 1,
"position" => 1
});
match coll.find(filter, Some(options)) {
Ok(cursor) => Ok(cursor),
Err(e) => err_as_string!(e),
}
Note: The code segment:
produces the BSON equivalent to the JSON object:
(more on that later)
doc! {
"team" => "BOS"
}
Step 1
{ "team": "BOS" }
- Select the database and collection
-
Set the query options
- "filter" → which documents to select
- "projection" (optional) → which fields from the document to return
Basic Steps to Query Data
let db = client.db("mlb");
let coll = db.collection("players");
let filter = Some(doc! { "team" => team });
let mut options = FindOptions::new();
options.projection = Some(doc! {
"_id" => 0,
"first_name" => 1,
"last_name" => 1,
"position" => 1
});
match coll.find(filter, Some(options)) {
Ok(cursor) => Ok(cursor),
Err(e) => err_as_string!(e),
}
Step 1
Step 2.2
Step 2.1
- Select the database and collection
- Set the query options
- "filter" → which documents to select
- "projection" (optional) → which fields from the document to return
- Check the result for a success
Basic Steps to Query Data
let db = client.db("mlb");
let coll = db.collection("players");
let filter = Some(doc! { "team" => team });
let mut options = FindOptions::new();
options.projection = Some(doc! {
"_id" => 0,
"first_name" => 1,
"last_name" => 1,
"position" => 1
});
match coll.find(filter, Some(options)) {
Ok(cursor) => Ok(cursor),
Err(e) => err_as_string!(e),
}
Step 1
Step 2.2
Step 2.1
Step 3
- Select the database and collection
- Set the query options
- "filter" → which documents to select
- "projection" (optional) → which fields from the document to return
- Check the result for a success
- Use the results as needed
Basic Steps to Query Data
let mut string = "{\"result\":[".to_owned();
for (i, doc_result) in cursor.enumerate() {
match json_string_from_doc_result(doc_result) {
Ok(json_string) => {
let new_string = if i == 0 {
json_string
} else {
format!(",{}"), json_string)
};
string.push_str(&new_string);
},
Err(e) => return e,
}
}
string.push_str("]}");
`Cursor`
implements
`Iterator`
For Instance:
The Three Core Tenets of Rust
-
Safety through the ownership system
-
Concurrency through core structs and features
-
Usability through standard traits and recursive macros
-
...and speed!
The Three Core Tenets of Rust
-
Safety through the ownership system
-
Concurrency through core structs and features
-
Usability through standard traits and recursive macros
Struct Referencing: Looks Good?
Ownership
Rust guarantees speed and safety at runtime by enforcing ownership and lifetimes at compile time.
Since the Client object is borrowed, its lifetime must predictably last at least as long as the database that contains it. ('a)
Arc
Atomic Reference Count is your friend.
How do we pass an Arc of self?
How do we pass an Arc of self?
Types and Traits.
The Three Core Tenets of Rust
-
Safety through the ownership system
-
Concurrency through core structs and features
-
Usability through standard traits and recursive macros
Mutability in Rust
By default, all variables in Rust are immutable.
However, mutability is a bit different in Rust...
Interior and Exterior Mutability
By default, all structs and variables in Rust have exterior immutability.
Immutable structures can still hold mutable components, as long as they follow the ownership system of Rust:
You may have one or the other of these two kinds of borrows, but not both at the same time:
-
one or more references (&T) to a resource.
-
exactly one mutable reference (&mut T).
Mutexes and RwLocks
One approach to guaranteeing these rules at compile time is to use RAII locks.
Connection Pools
How we learned to handle mutability,
the hard way.
Traditional Capped Pool
Fixed-length arrays of locks and sockets.
Acquire socket lock, Use socket, Release socket lock.
Hyper
Explicit extraction of lock-free sockets!
Lock the pool
Pop a stream (S)
Return the stream
with a pool reference
Our Friend, the Condvar
Master pool lock with variable-length, lock-free sockets.
If no sockets are available and we are capped at the number of open sockets, wait on the condition variable until we've been repopulated.
Thank you, Condvar.
The Three Core Tenets of Rust
-
Safety through the ownership system
-
Concurrency through core structs and features
-
Usability through standard traits and recursive macros
Recursive Macros
The Dark Ages
Not very readable!
let mut update = bson::Document::new();
let mut set = bson::Document::new();
set.insert("director".to_owned(), Bson::String("Robert Zemeckis".to_owned()));
update.insert("$set".to_owned(), Bson::Document(set));
First try
Do we really need a second macro?
let update = doc! {
"$set" => nested_doc! {
"director" => Bson::String("Robert Zemeckis".to_owned())
}
};
#[macro_export]
macro_rules! doc {
( $( $k:expr => $v: expr),* ) => {
{
let mut doc = Document::new();
$(
doc.insert($k.to_owned(), $v);
)*
doc
}
};
}
#[macro_export]
macro_rules! nested_doc {
( $( $k:expr => $v: expr),* ) => {
Bson::Document(doc!(
$( $k => $v),*
))
}
}
Not quite there yet...
Explicit types still needed
let update = doc! {
"$set" => {
"director" => Bson::String("Robert Zemeckis".to_owned())
}
};
#[macro_export]
macro_rules! add_to_doc {
($doc:expr, $key:expr => ($val:expr)) => {{
$doc.insert($key.to_owned(), $val);
}};
($doc:expr, $key:expr => [$($val:expr),*]) => {{
let vec = vec![$($val),*];
$doc.insert($key.to_owned(), Bson::Array(vec));
}};
($doc:expr, $key:expr => { $($k:expr => $v:tt),* }) => {{
$doc.insert($key.to_owned(), Bson::Document(doc! {
$(
$k => $v
),*
}));
}};
}
#[macro_export]
macro_rules! doc {
( $($key:expr => $val:tt),* ) => {{
let mut document = Document::new();
$(
add_to_doc!(document, $key => $val);
)*
document
}};
}
Ahh...much better!
let doc1 = doc! { "tags" => ["a", "b", "c"] };
let doc2 = doc! { "tags" => ["a", "b", "d"] };
let doc3 = doc! { "tags" => ["d", "e", "f"] };
coll.insert_many(vec![doc1.clone(), doc2.clone(), doc3.clone()], false, None)
.ok().expect("Failed to execute insert_many command.");
// Build aggregation pipeline to unwind tag arrays and group distinct tags
let project = doc! { "$project" => { "tags" => 1 } };
let unwind = doc! { "$unwind" => ("$tags") };
let group = doc! { "$group" => { "_id" => "$tags" } };
#[macro_export]
macro_rules! bson {
([$($val:tt),*]) => {{
let mut array = Vec::new();
$(
array.push(bson!($val));
)*
$crate::Bson::Array(array)
}};
([$val:expr]) => {{
$crate::Bson::Array(vec!(::std::convert::From::from($val)))
}};
({ $($k:expr => $v:tt),* }) => {{
$crate::Bson::Document(doc! {
$(
$k => $v
),*
})
}};
($val:expr) => {{
::std::convert::From::from($val)
}};
}
#[macro_export]
macro_rules! doc {
() => {{ $crate::Document::new() }};
( $($key:expr => $val:tt),* ) => {{
let mut document = $crate::Document::new();
$(
document.insert($key.to_owned(), bson!($val));
)*
document
}};
}
Auto-converted types!
and a little bit about traits...
High focus for Rust 1.0: Extendability
- Get core features down, provide traits for the community.
- Large portion of core rust features rely on generic traits.
- Flexible functional-style programming in a largely procedural language.
a few useful traits...
-
Conversion Traits: Auto-convert types.
-
Encoder Traits: Auto-encode structs.
-
Error Traits: Handle errors unobtrusively.
-
Dereference Traits: Utilize the auto-dereferencing system.
-
Iterator Traits: Iterate naturally over custom structs.
-
I/O Traits: Perform I/O ops over custom structs.
- ...and many, many more.
The Three Core Tenets of Rust
-
Safety through the ownership system
-
Concurrency through core structs and features
-
Usability through standard traits and recursive macros
The Three Core Tenets of Rust
...and speed!
Other Rust language features:
Pattern Matching/Enum types
Suppose you had the following Python function...
def sum_of_first_three_elements(num_list):
if len(num_list) < 3:
return None
return sum(num_list[:3])
How would you implement that in Rust?
The second part is easy...
# Python
sum(num_list[:3])
// Rust
num_list[0] + num_list[1] + num_list[2]
But what would the return type be?
There's no "null" in Rust!
Solution: Optional types
pub enum Option<T> {
None,
Some(T),
}
fn sum_of_first_three_elements(num_list: &[i32]) -> Option<i32> {
if num_list.len() < 3 {
None
} else {
Some(num_list[0] + num_list[1] + num_list[2])
}
}
Example function implemented in Rust
It might return an int...but it might not
"variants": almost like constructors
"enumerated type"
note: "T" is the generic type of the parameter
Obvious question: But how do you use options?
Not-so obvious answer (unless you've done functional programming before): pattern matching!
Pattern matching
Like switch statements...but actually useful!
General form
match <expression> {
<first pattern> => <resulting expression>,
<second pattern> => <resulting expression>,
...
// Optional catch-all
_ => <resulting expression>
}
Example: pattern matching an option
match maybe_an_int {
Some(i) => println!("The integer is: {}", i),
None => println!("There is no integer :(")
}
"match" is actually an expression
fn say_hey(name: Option<&str>) {
let person = match name {
Some(string) => string,
None => "you"
};
println!("Hey, {}!", person);
}
say_hey(Some("Joe"); // prints "Hey, Joe!"
say_hey(None); // prints "Hey, you!"
Advanced pattern matching
pub enum Bson {
FloatingPoint(f64),
I32(i32),
I64(i64),
Binary(BinarySubtype, Vec<u8>),
// other types omitted
}
match bson_document.get("some_key") {
// Explicit fallthrough, and ignored capture of single variable
Some(&Bson::I32(_)) |
Some(&Bson::I64(_)) => println("I got an int!"),
// Multi-line results
Some(&Bson::FloatingPoint(f)) => {
let i = f as i32;
println!("I didn't get an int...but if I did, it would look like this: {}", f);
}
// Ignored capture of multiple variables
Some(&Bson::Binary { .. }) => println!("I got some bits"),
// "wildcard" matches everything else
_ => println!("I got a rock...")
}
Benefits of pattern matching
- Flexible
- Completely safe
- If you miss a variant (without specifying a wild card), the compiler will tell you
- Really freaking fast
Bonus time
You can implement methods on enums just like other structs!
impl Bson {
fn print_value(&self) {
match self {
Bson::I32(i) => println!("{}", i),
Bson::I64(i) => println!("{}", i),
Bson::FloatingPoint(f) => println!("{}", f),
...
}
}
}
let bson_int = Bson::I32(17);
bson_int.print_value(); // prints "17"
Driver Status
Complete ✓
-
bson library
-
OID generation
-
stable macro
-
- connection strings
-
wire protocol
-
CRUD, commands, and cursors
-
bulk writes
-
connection pooling
-
GridFS
-
server discovery and monitoring (SDAM)
-
command monitoring (APM)
-
SCRAM-SHA-1 auth
-
server selection & failover
Up Next
-
shard tagging and other server commands
-
SSL Support
-
Other auth mechanisms
-
__
Learn more...
...and join us!
Questions?
The New MongoDB Rust Driver
By Kevin Yeh
The New MongoDB Rust Driver
- 148