Extending Python with Rust for performance

@swissgathu

SPEED

RELIABILITY

a rust primer

Rust is a systems  language that pursues the trifecta of:

  • speed

  • safety

  • concurrency

Speed

  • No Garbage Collection
  • LLVM
  • Zero cost abstractions
  • Minimal Runtime
  • Compile time generics monomorphization

Safety

  • Memory safety
    • You never explicitly free memory.
  • Automatic Resource Management
    • You never explicitly release resources.

 

Rust means never having to close a socket
~ Yehuda Katz

Concurrency

  • Fearless Concurrency

    • Threads.
    • Message Passing.
    • Shared State.
    • Extensible Concurrency
fn main() {
    for count in 0..3 {
        println!("{}. Welcome to Andela!", count);
    }
}
pub trait Summarizable {
    fn summary(&self) -> String;
}
pub struct Tweet {
    pub username: String,
    pub content: String,
    pub reply: bool,
    pub retweet: bool,
}

impl Summarizable for Tweet {
    fn summary(&self) -> String {
        format!("{}: {}", self.username, self.content)
    }
}

TRAITS

virtualenv & pip & distutils & setuptools

rustup & cargo

rustup

the rust toolchain manager

cargo

the rust package manager

HOW

Foreign Function Interface

PYTHON

C ABI + header (rust) & CFFI (python)

RUST

Setup Tools Extension

def build_native(spec):
    """Build rust binary"""
    # Step 1: build the rust library
    build = spec.add_external_build(
        cmd=['cargo', 'build', '--release'],path='./rust'
    )

    # Step 2: add a cffi module based on the dylib we built
    spec.add_cffi_module(
        module_path='native._native',
        dylib=....,
        header_filename=...,
        rtld_flags=['NOW', 'NODELETE']
    )

python setup.py bdist_wheel

csvterm-0.1-py2.py3-none-macosx_10_12_x86_64.whl

Sample Code






def tail(file, number, select, delimiter):
    """Show the last N rows of CSV data. Default is 10 rows"""
    ....

Python Function





// Show the last N rows of CSV data. Default is 10 rows
fn rtail(fname: &str, 
         select: Option<Vec<usize>>, 
         delimiter: u8, 
         count: usize,) -> Result<Vec<u8>, Box<Error>> {
    
    ...
}

Rust function

#[no_mangle]
pub unsafe extern "C" fn tail(
    fname: *const c_char,
    select: *const c_char,
    delimiter: c_char,
    count: i32,
) -> *const u8 {
    ...
    let v = rtail(&fname, select, delimiter as u8, count).unwrap();
    ...
}

Rust C ABI

from . import _native

tail_bytes = _native.lib.tail(str.encode(filename),
                              str.encode(filter_cols_nums),
                              str.encode(delimiter),
                              number)
tail_string = ffi.string(bytes).decode()

tail_rows = tail_string.split('\n')

Python calling Rust

RESULTS

SPEED

MEMORY

Pure Python

Python + Rust

Still Interested?

rust-lang.org

doc.rust-lang.org/book/

Rust Nairobi

Made with Slides.com