Distributed Systems &

Rust

Dan Burkert

irc: dcb

GitHub: danburkert

Distributed Systems

• IO & Networking
  • protocols
  • abstractions & threading models
• Operations
  • monitoring
  • tracing
  • debugging
• Security & Encryption
• Performance
• ...many more...

std::net

mio

networking in Rust

// accept TCP connections and process them,
// spawning a new thread for each one
for stream in listener.incoming() {
    match stream {
        Ok(stream) => {
            thread::spawn(move|| {
                // connection succeeded
                handle_client(stream)
            });
        },
        Err(e) => handle_failure(e),
    }
}

fn ready(&mut self,
         event_loop: &mut EventLoop<Echo>,
         token: Token,
         events: EventSet) {

    if events.is_readable() {
        match token {
            SERVER => self.server.accept(event_loop),
            CLIENT => self.client.readable(event_loop),
            i => self.server.conn_readable(event_loop, i),
        }
    }

    if events.is_writable() {
        match token {
            SERVER => panic!("received writable for token 0"),
            CLIENT => self.client.writable(event_loop),
            _ => self.server.conn_writable(event_loop, token),
        };
    }
}

non-blocking sockets

invert

program flow

let's build something

a simple key value store

a simple key value store

$ telnet 127.0.0.1 5556
...
> PUT my-key some-value
OK
> GET my-key
some-value
> GET other-key
NONE
> bogus command
ERR

github.com/danburkert/simple-kv

/// A simple key value database server.
///
/// The server listens for text-based TCP
/// messages in the following formats:
///
/// * 'PUT <key> <value>'
/// * 'GET <key>'
pub struct Server {
    listener: TcpListener,
    connections: Slab<Connection>,
    db: HashMap<String, String>,
}

a simple key value store

impl Handler for Server {

    fn ready(&mut self, event_loop: &mut EventLoop<Server>, token: Token, events: EventSet) {

        if token == LISTENER {
            self.accept_connections(event_loop);
        } else {

            if events.is_readable() {
                if let Err(error) = self.connection_readable(token) {
                    self.connections.remove(token);
                    return;
                }
            }

            if events.is_writable() {
                if let Err(error) = self.connections[token].writable() {
                    self.connections.remove(token);
                    return
                }
            }

            let events = self.connections[token].events;
            if let Err(error) = event_loop.reregister(&mut self.connections[token].socket,
                                                      token,
                                                      events, PollOpt::edge() | PollOpt::oneshot()) {
                self.connections.remove(token);
            }
        }
    }
}

Handling the Event Loop

struct Connection {

    socket: TcpStream,

    /// Holds bytes read from the socket,
    /// but not yet deserialized.
    read_buf: Vec<u8>,

    /// Holds a stream of bytes to be
    /// sent to the socket on the next
    /// writable event.
    write_buf: Vec<u8>,

    /// The set of events which the
    /// connection is interested in handling.
    events: EventSet,
}

Buffer Management

impl Connection {

    /// Called when there are bytes available
    /// to read on the connection's socket.
    ///
    /// Reads all available bytes, and then scans
    /// the bytes for newline characters. For each
    /// newline character, the corresponding line
    /// is deserialized into a message.
    ///
    /// Returns all of the messages read.
    fn readable(&mut self) -> Result<Vec<Message>> {
        let mut messages = Vec::new();
        let read_buf = &mut self.read_buf;

        let read_from = read_buf.len();
        match self.socket.read_to_end(read_buf) {
            Ok(0) => return Ok(messages),
            Ok(_) => (),
            Err(ref error) if error.kind() == ErrorKind::WouldBlock => (),
            Err(error) => return Err(error),
        }

        let mut lo = 0;
        // Check the newly read bytes for line seperators. For each line, decode it
        // into a message and add it to the messages list.
        for (hi, &c) in read_buf[read_from..].iter().enumerate() {
            if c == '\n' as u8 {
                let line = &read_buf[lo..hi];
                messages.push(Message::from_bytes(line));
                lo = hi + 1;
            }
        }

        // Remove bytes that have been decoded into lines.
        read_buf.drain(..lo).count();
        Ok(messages)
    }
}

Protocols & Serialization

#[derive(Debug)]
enum Message {
    /// A get message, including the key to look up.
    Get(String),
    /// A put message, including the key and value.
    Put(String, String),
    /// Unable to decode the message
    Error,
}

impl Message {
    fn from_bytes(bytes: &[u8]) -> Message {
        let line = match str::from_utf8(bytes) {
            Ok(chars) => chars,
            Err(..) => return Message::Error,
        };

        let words: Vec<&str> = line.split_whitespace().collect();
        let len = words.len();
        if len < 1 { return Message::Error }
        match words[0] {
            "GET" => {
                if len != 2 { Message::Error }
                else { Message::Get(words[1].to_owned()) }
            },
            "PUT" => {
                if len != 3 { Message::Error }
                else { Message::Put(words[1].to_owned(), words[2].to_owned()) }
            },
            _ => { words[0]); Message::Error},
        }
    }
}

Messages

why non-blocking

• scalability
• (lack of) synchronization
• abstractions
  • ​futures & streams
  • coroutines

Distributed Systems

• IO & Networking
  • protocols
  • abstractions & threading models
• Operations
  • monitoring
  • tracing
  • debugging
• Security & Encryption
• Performance
• ...many more...

Operations

• metrics
• reporting
• tracing
• debugging

Special Thanks To

rust-lang/rust

carllerche/mio

hoverbear/raft