Extending Python with Rust for performance
@swissgathu
SPEED
RELIABILITY
a rust primer
Rust is a systems language that pursues the trifecta of:
-
speed
-
safety
-
concurrency
Speed
- No Garbage Collection
- LLVM
- Zero cost abstractions
- Minimal Runtime
- Compile time generics monomorphization
Safety
-
Memory safety
- You never explicitly free memory.
-
Automatic Resource Management
- You never explicitly release resources.
Rust means never having to close a socket
~ Yehuda Katz
Concurrency
-
Fearless Concurrency
- Threads.
- Message Passing.
- Shared State.
- Extensible Concurrency
fn main() {
for count in 0..3 {
println!("{}. Welcome to Andela!", count);
}
}
pub trait Summarizable {
fn summary(&self) -> String;
}
pub struct Tweet {
pub username: String,
pub content: String,
pub reply: bool,
pub retweet: bool,
}
impl Summarizable for Tweet {
fn summary(&self) -> String {
format!("{}: {}", self.username, self.content)
}
}
TRAITS
virtualenv & pip & distutils & setuptools
rustup & cargo
rustup
the rust toolchain manager
cargo
the rust package manager
HOW
Foreign Function Interface
PYTHON
C ABI + header (rust) & CFFI (python)
RUST
Setup Tools Extension
def build_native(spec):
"""Build rust binary"""
# Step 1: build the rust library
build = spec.add_external_build(
cmd=['cargo', 'build', '--release'],path='./rust'
)
# Step 2: add a cffi module based on the dylib we built
spec.add_cffi_module(
module_path='native._native',
dylib=....,
header_filename=...,
rtld_flags=['NOW', 'NODELETE']
)
python setup.py bdist_wheel
csvterm-0.1-py2.py3-none-macosx_10_12_x86_64.whl
Sample Code
def tail(file, number, select, delimiter):
"""Show the last N rows of CSV data. Default is 10 rows"""
....
Python Function
// Show the last N rows of CSV data. Default is 10 rows
fn rtail(fname: &str,
select: Option<Vec<usize>>,
delimiter: u8,
count: usize,) -> Result<Vec<u8>, Box<Error>> {
...
}
Rust function
#[no_mangle]
pub unsafe extern "C" fn tail(
fname: *const c_char,
select: *const c_char,
delimiter: c_char,
count: i32,
) -> *const u8 {
...
let v = rtail(&fname, select, delimiter as u8, count).unwrap();
...
}
Rust C ABI
from . import _native
tail_bytes = _native.lib.tail(str.encode(filename),
str.encode(filter_cols_nums),
str.encode(delimiter),
number)
tail_string = ffi.string(bytes).decode()
tail_rows = tail_string.split('\n')
Python calling Rust
RESULTS
SPEED
MEMORY
Pure Python
Python + Rust
Still Interested?
rust-lang.org
doc.rust-lang.org/book/
Rust Nairobi
Extending Python with Rust for Performance
By Matt Gathu
Extending Python with Rust for Performance
- 1,088