Rusty Python

Rust, a new language for our programmer tool belt

Matteo Bertini

@naufraghi

PyCon Otto | 6-9 April 2017 | Florence

Rust

  • Born as a personal project started in 2006 by Mozilla employee Graydon Hoare
  • Mozilla began sponsoring the project in 2009 and announced it in 2010
  • A young language, with a growing usage in production (Dropbox, Mozilla, Canonical, OVH, ...)
$ cargo new hello-world --bin
     Created binary (application) `hello-world` project
$ cd hello-world/
$ tree
.
├── Cargo.toml
└── src
    └── main.rs

1 directory, 2 files

Hello, World!

install the Rust toolchain with `rustup`

Hello, World!

fn main() {
    println!("Hello, World!");
}

huhu? `println!`?

How we run it?

`cargo run`

$ cargo run
   Compiling hello-world v0.1.0 (file:///home/naufraghi/.../hello-world)
    Finished dev [unoptimized + debuginfo] target(s) in 0.41 secs
     Running `target/debug/hello-world`
Hello world!

Docstrings?

//! Programma di esempio per _PyCon Otto_,
//! l'immancabile «Hello, World!».
fn main() {
    println!("Hello, World!");
}

`cargo doc --open`

Integrated docs

Cool, but in Python have `doctest`, I can't live without it!

And Rust "borrowed" the idea!

//! Libreria di esempio per _PyCon Otto_,
//! l'immancabile «Hello, World!».
//!
//! ```
//! use hello_world_lib::greater;
//! let message = greater("World");
//! assert_eq!(message, "Hello, World!");
//! ```

pub fn greater(name: &str) -> String {
    format!("Hello, {}!", name)
}

in fact a lot of ideas from Python

Run `cargo test --doc`

$ cargo test --doc
    Finished dev [unoptimized + debuginfo] target(s) in 0.0 secs
   Doc-tests hello-world-lib

running 1 test
test greater_0 ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured

Let me see that code again...

pub fn greater(name: &str) -> String {
    format!("Hello, {}!", name)
}

Where is the `return`?

It's implicit, the last expression is returned

... beware of forgetting some `return` in Python after a switch :)

Optional arguments

pub fn greater2(optional_name: Option<&str>) -> String {
    //! Restituisce un saluto di default se il nome è None
    //!
    //! ```
    //! use hello_world_lib::greater2;
    //! let message = greater2(None);
    //! assert_eq!(message, "Hello, World!");
    //!
    //! let message = greater2(Some("Matteo"));
    //! assert_eq!(message, "Hello, Matteo!");
    //! ```
    let name = match optional_name {
        Some(name) => name,
        None => "World",
    };
    format!("Hello, {}!", name)
}

no default values... yet

`if` and `match`are expressions in Rust

Managing None

var = some_optional_value()
if var is not None:
   res = do_something_with(var)
else:
   res = None
res = if let Some(var) = some_optional_value() {
   Some(do_something_with(var))
} else {
   None
}

Python

Rust

`None` is a serious thing

fn main() {
    let out = if let Some(value) = some_optional_value("some") {
        do_something_with(value)
    } else {
        None
    };
    println!("out: {:?}", out);
}

Nice error messages

Rust Playground

In Rust we have other ways

some_optional_value().map(do_something_with)
pub enum Option<T> {
    None,
    Some(T),
}

`Option::map` will not call `do_something_with` if the value is `None`

`Option` docs

Methods

struct Circle {
    x: f64,
    y: f64,
    radius: f64,
}

impl Circle {
    fn area(&self) -> f64 {
        std::f64::consts::PI * (self.radius * self.radius)
    }
}
self  |  &self  |  &mut self

Traits

struct Circle {
    x: f64,
    y: f64,
    radius: f64,
}

trait HasArea {
    fn area(&self) -> f64;
}

impl HasArea for Circle {
    fn area(&self) -> f64 {
        std::f64::consts::PI * (self.radius * self.radius)
    }
}

this is what your lib users will need to implement

Duck typing

def print_area(shape) {
    print("This shape has an area of {}".format(shape.area()));
}

Traits are something we can use where in Python we were using duck typing

fn print_area<T: HasArea>(shape: T) {
    println!("This shape has an area of {}", shape.area());
}

Python

Rust

Implementing Traits

struct Counter {
    count: usize,
}
impl Counter {
    fn new() -> Counter {
        Counter { count: 0 }
    }
}
impl Iterator for Counter {
    type Item = usize;
    fn nxt(&mut self) -> Option<Self::Item> {
        self.count += 1;
        Some(self.count)
    }
}

Traits are somewhat similar to the `collections.abc` in Python

class Counter(abc.Iterator):
    def __init__(self):
        self.count = 0
    def __nxt__(self):
        self.count += 1
        return self.count  

with a typo in both

Implementing Traits

error[E0407]: method `nxt` is not a member of trait `Iterator`
  --> <anon>:11:5
   |
11 |       fn nxt(&mut self) -> Option<Self::Item> {
   |  _____^ starting here...
12 | |         self.count += 1;
13 | |         Some(self.count)
14 | |     }
   | |_____^ ...ending here: not a member of trait `Iterator`

error[E0046]: not all trait items implemented, missing: `next`
  --> <anon>:9:1
   |
9  |   impl Iterator for Counter {
   |  _^ starting here...
10 | |     type Item = usize;
11 | |     fn nxt(&mut self) -> Option<Self::Item> {
12 | |         self.count += 1;
13 | |         Some(self.count)
14 | |     }
15 | | }
   | |_^ ...ending here: missing `next` in implementation
   |
   = note: `next` from trait: `fn(&mut Self) ->
           std::option::Option<<Self as std::iter::Iterator>::Item>`
>>> c = Counter()
Traceback (most recent call last):
  File "<ipython-input-29-f3cf168481b9>", line 1, in <module>
    c = Counter()
TypeError: Can't instantiate abstract class Counter with abstract
           methods __next__

Runtime check

Compile time check

We have barely seen Rust as a language

  • Rust is not easy,
  • there are a lot of concepts,
  • one can learn step-by-step.
  • The tooling is good
  • The docs are good
  • Let's use Rust!

Ownership

Lifetimes

Mutability

Macros

Pattern Matching

Let's rewrite Electric Sheep

Fly down!

Paper on http://flam3.com/flame_draves.pdf, 41 pages

         / a b \   / x \   / e \
f(x,y) = |     | · |   | + |   |
         \ c d /   \ y /   \ f /

Given 4 linear function with some magic parameters (a,b,c,d,e,f)

Choose a random starting point, say (0,0)

Loop:

  • pick a random function
  • apply the func on the last point

  • collect the result

Iterated Function System

Barnsley fern

Iterated Function System

F_{1}(x, y) \sim w_1
F1(x,y)w1F_{1}(x, y) \sim w_1
F_{2}(x, y) \sim w_2
F2(x,y)w2F_{2}(x, y) \sim w_2
F_{3}(x, y) \sim w_3
F3(x,y)w3F_{3}(x, y) \sim w_3
F_{4}(x, y) \sim w_4
F4(x,y)w4F_{4}(x, y) \sim w_4
p_{i+1} = F_{k}(p_i)
pi+1=Fk(pi)p_{i+1} = F_{k}(p_i)

Plot the histogram

Take one at random and collect the points

Python

for i in range(iterations):
    f = np.random.choice(arange(4), p=weights)

    if f == 0:
        P = F1 * P + F1c
    elif f == 1:
        P = F2 * P + F2c
    elif f == 2:
        P = F3 * P + F3c
    else:
        P = F4 * P + F4c

    if i > pointsToSkip:
        drawPoints[frame][i-pointsToSkip] = squeeze(asarray(P))

Rust

for i in 0..iterations + 1 {
    let f = wc.ind_sample(&mut rng);
    if f == 1 {
        p = f1 * p + f1c;
    } else if f == 2 {
        p = f2 * p + f2c;
    } else if f == 3 {
        p = f3 * p + f3c;
    } else {
        p = f4 * p + f4c;
    }
    if i > points_to_skip {
        points.push((p.x, p.y));
    }
}

How does it work?

Build the extension

$ python setup.py develop 
running develop
running egg_info
writing top-level names to ifs.egg-info/top_level.txt
writing dependency_links to ifs.egg-info/dependency_links.txt
writing ifs.egg-info/PKG-INFO
reading manifest file 'ifs.egg-info/SOURCES.txt'
writing manifest file 'ifs.egg-info/SOURCES.txt'
running build_ext
running build_rust
cargo rustc --lib --manifest-path Cargo.toml \
            --features cpython/extension-module cpython/python3-sys \
            --release -- --crate-type cdylib
    Finished release [optimized] target(s) in 0.0 secs

Creating /home/naufraghi/.../python3.5/site-packages/ifs.egg-link (link to .)
ifs 1.0 is already the active version in easy-install.pth

Installed /home/naufraghi/Documents/src/rusty-python-pyconotto/barnsley-fern
Processing dependencies for ifs==1.0
Finished processing dependencies for ifs==1.0

Generating the fern

$ python fern.py
Using Python
Frame 0 generated
...
Frame 11 generated
Fern data generation completed in 23.3s
Rendered frame 0000
...
Rendered frame 0011
Converting to animated gif...
Gif Generated

Generating the fern

$ python fern.py rust
Using Rust
Frame 0 generated
...
Frame 11 generated
Fern data generation completed in 0.4s
Rendered frame 0000
...
Rendered frame 0011
Converting to animated gif...
Gif Generated

Generating the fern

$ python fern.py rust
Using Rust
Frame 0 generated
...
Frame 11 generated
Fern data generation completed in 3.3s
Rendered frame 0000
...
Rendered frame 0011
Converting to animated gif...
Gif Generated
  • image size 300 -> 600
  • iterations 50000 -> 500000

Python wrapper

use cpython::{PyResult, Python};
use barnsley_fern;

py_module_initializer!(_iterated_function_systems,
                       _inititerated_function_systems,
                       PyInit__iterated_function_systems,
                       |py, m| {
    try!(m.add(py, "__doc__", "Rust iterated function systems"));
    try!(m.add(py,
               "barnsley_fern",
               py_fn!(py,
                      py_barnsley_fern(iterations: usize,
                                       points_to_skip: usize,
                                       bending: f64))));
    Ok(())
});

fn py_barnsley_fern(py: Python,
                    iterations: usize,
                    points_to_skip: usize,
                    bending: f64)
                    -> PyResult<Vec<(f64, f64)>> {
    Ok(barnsley_fern(iterations, points_to_skip, bending))
}

Python wrapper

rust-cpython has an impl for most builtin types

setup.py

Cargo.toml

from setuptools import setup
from setuptools_rust import RustExtension

setup(name='ifs',
      version='1.0',
      rust_extensions=[
          RustExtension('ifs._iterated_function_systems',
                        'Cargo.toml', debug=False)],
      packages=['ifs'],
      # rust extensions are not zip safe, just like C-extensions.
      zip_safe=False)
[package]
name = "ifs"
version = "0.1.0"
authors = ["Matteo Bertini <matteo@naufraghi.net>"]

[lib]
name = "iterated_function_systems"
crate-type = ["cdylib"]

[dependencies]
nalgebra = "0.11.2"
rand = "0.3.15"
cpython = "0.1"

CI up and running

[tox]
envlist = py27, py35

[testenv]
commands =
    python setup.py install
    pip install example/
    py.test
deps =
    pytest
matrix:
  allow_failures:
    os: osx
  include:
    - language: generic
      os: osx
      python: 2.7
      env:
        - RUST_VERSION=stable
    - language: python
      python: 2.7
      env:
        - RUST_VERSION=stable
    ...

#  Manually install python on osx
before_install:
  - if [[ $TRAVIS_OS_NAME == 'osx' ]]; then ci/osx-install.sh; fi

install:
  - curl https://sh.rustup.rs -sSf | sh -s -- -y --default-toolchain $RUST_VERSION
  - export PATH="$HOME/.cargo/bin:$PATH"
  - rustc -V
script:
  - sudo pip install tox
  - tox

Take away

  • Rust seems to be here to stay
  • Rust is high level enough
  • Integrating Rust in Python is easy
  • We can write fast and safe Rust extensions!

Thanks!

Questions?

Other choices

  • Cython
  • C: CPython / ffi
  • c++: SIP (PyQt)
  • c++: pybind11 (modern boost::python)
  • fortran: f2py
  • Rust: CPython / ffi

Resources

  • https://www.rust-lang.org/
  • https://play.rust-lang.org/
  • https://doc.rust-lang.org/book/
  • https://github.com/dgrunwald/rust-cpython
  • https://github.com/fafhrd91/setuptools-rust
  • http://flam3.com/flame_draves.pdf
  • https://github.com/marceloramires/FractalRendering

Rusty Python | PyCon Otto | 6-9 Aprile 2017

By Matteo Bertini

Rusty Python | PyCon Otto | 6-9 Aprile 2017

Rust, a new language for our programmer tool belt

  • 229