Building JSON-RPC APIs in Rust

Tomasz Drwięga

@tomusdrw

Parity Technologies


25 July 2018, Wrocław Rust Meetup

https://slides.com/tomaszdrwiega/jsonrpc

JSON-RPC

{
  "jsonrpc": "2.0",
  "id": 1,
  "method": "getTransactions",
  "params": [{ "since": 1532502205406 }]
}
  • Stateless
  • Transport agnostic
  • Remote Procedure Call protocol

Request

JSON-RPC

{
  "jsonrpc": "2.0",
  "id": 1,
  "result": []
}

Response

{
  "jsonrpc": "2.0",
  "id": 1,
  "error": {
    "code": 32000,
    "message": "Something went wrong",
    "data": ["Validation error 1"]
  }
}

Why JSON-RPC?

  • Simple
  • Transport agnostic
  • Support in various languages
  • Standardized errors
  • Standard in crypto-space
    (Bitcoin, Ethereum)

What we do at Parity?

  • Parity Ethereum - Ethereum Client
  • Parity Bitcoin - Bitcoin Client
  • WASM (wasmi)
  • Whisper
  • IPFS / libp2p
  • Polkadot - The Internet of Blockchains
  • Parity Substrate
  • Many RPCs exposed :)
     

Everything built with Rust (+ some JS).

crates.io/jsonrpc-core

  • Rust library to easily build JSON-RPCs
  • Core + Transports
  • Async support (futures)
  • Serde-based
  • Couple of useful extensions

Dude, show the code!

extern crate jsonrpc_core as rpc;

#[test]
fn should_send_hello() {
    // given
    let mut io = rpc::IoHandler::new();
    io.add_method("say_hello", |_params: rpc::Params| {
        Ok(rpc::Value::String("hello".to_string()))
    });
    
    
    // when
    let response = io.handle_request_sync(r#"
        {
         "jsonrpc": "2.0",
         "id": 1,
         "method":"say_hello"
        }
    "#);
    
    // then
    assert_eq!(response, Some(
        r#"{"jsonrpc":"2.0","result":"hello","id":1}"#.into()
    ));
}

Transports

  • HTTP (hyper-based)
  • MiniHTTP (tokio-minihttp)
  • WebSockets (rewrite upcoming)
  • IPC (Windows & *nix)
  • TCP

HTTP server

extern crate jsonrpc_core as rpc;
extern crate jsonrpc_http_server as http;

fn main() {
    let mut io = rpc::IoHandler::new();
    io.add_method("say_hello", |_params: rpc::Params| {
        Ok(rpc::Value::String("hello".to_string()))
    });

    let address = "127.0.0.1:3030".parse()
        .expect("Valid address given");

    let server = http::ServerBuilder::new(io)
        .threads(3)
        .start_http(&address)
        .expect("Server should start");

    server.wait()
}

Microbenchmarks

# JSON-RPC minihttp (3 threads)
$ cargo run
$ wrk -t 4 -c 4 http://localhost:3030

Running 10s test @ http://localhost:3030
  4 threads and 4 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   183.91us   91.04us   3.45ms   90.69%
    Req/Sec     5.50k     1.64k    7.73k    53.96%
  221195 requests in 10.10s, 44.93MB read
  Non-2xx or 3xx responses: 221195
Requests/sec:  21900.60
Transfer/sec:      4.45MB


# Node
$ ./node.js
$ wrk -t 4 -c 4 http://localhost:3000

Running 10s test @ http://localhost:3000
  4 threads and 4 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   150.06us  201.16us  10.71ms   98.54%
    Req/Sec     7.05k     0.95k    8.53k    88.12%
  283229 requests in 10.10s, 37.55MB read
Requests/sec:  28042.79
Transfer/sec:      3.72MB

Microbenchmarks

# JSON-RPC minihttp (3 threads)
$ cargo run --release
$ wrk -t 4 -c 4 http://localhost:3030

Running 10s test @ http://localhost:3030
  4 threads and 4 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    53.57us   54.91us   3.12ms   99.46%
    Req/Sec    19.05k     2.35k   23.99k    67.57%
  765634 requests in 10.10s, 155.53MB read
  Non-2xx or 3xx responses: 765634
Requests/sec:  75812.61
Transfer/sec:     15.40MB


# JSON-RPC minihttp (3 threads) with payload
$ cargo run --release
$ wrk -t 4 -c 4 --script ./script.lua http://localhost:3030

Running 10s test @ http://localhost:3030
  4 threads and 4 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    74.00us   42.98us   2.85ms   98.49%
    Req/Sec    13.52k     1.50k   15.59k    52.72%
  543735 requests in 10.10s, 87.63MB read
Requests/sec:  53835.55
Transfer/sec:      8.68MB

jsonrpc-macros

extern crate jsonrpc_core as rpc;
#[macro_use]
extern crate jsonrpc_macros;

use rpc::Result;
build_rpc_trait! {
	pub trait Rpc {
		/// Adds two numbers and returns a result
		#[rpc(name = "add")]
		fn add(&self, u64, u64) -> Result<u64>;
	}
}

pub struct RpcImpl;
impl Rpc for RpcImpl {
	fn add(&self, a: u64, b: u64) -> Result<u64> {
		Ok(a + b)
	}
}

fn main() {
    let mut io = rpc::IoHandler::new();
    io.extend_with(RpcImpl.to_delegate());
}

async

extern crate jsonrpc_core as rpc;
#[macro_use]
extern crate jsonrpc_macros;

use rpc::{BoxFuture, futures::{self, Future}};

build_rpc_trait! {
    pub trait Rpc {
        /// Adds two numbers and returns a result
        #[rpc(name = "add")]
	fn add(&self, u64, u64) -> BoxFuture<u64>;
    }
}

pub struct RpcImpl;
impl Rpc for RpcImpl {
     fn add(&self, a: u64, b: u64) -> BoxFuture<u64> {
        let (tx, rx) = futures::oneshot();
        ::std::thread::spawn(move || {
            tx.send(a + b)
        });
        Box::new(
            rx.map_err(|_| unreachable!())
        )
    }
}

Extensions

  • Metadata
    (extract transport-specific data)
  • Middlewares (intercept calls)
  • Test Utils (no Strings in tests)
  • Publish-Subscribe
    (subscribe to notifications)

PubSub

extern crate jsonrpc_core as rpc;
extern crate jsonrpc_pubsub as pubsub;
#[macro_use]
extern crate jsonrpc_macros as macros;
use rpc::Result;

build_rpc_trait! {
    pub trait Rpc {
      type Metadata;

      #[pubsub(name = "hello")] {
	/// Hello subscription
	#[rpc(name = "hello_subscribe", alias = ["hello_sub", ])]
	fn subscribe(&self, Self::Metadata, macros::pubsub::Subscriber<String>, u64);

	/// Unsubscribe from hello subscription.
	#[rpc(name = "hello_unsubscribe")]
	fn unsubscribe(&self, pubsub::SubscriptionId) -> Result<bool>;
      }
    }
}
#[derive(Default)]
struct RpcImpl {
    uid: atomic::AtomicUsize,
    active: Arc<RwLock<HashMap<pubsub::SubscriptionId, macros::pubsub::Sink<String>>>>,
}
impl Rpc for RpcImpl {
    fn subscribe(&self, _meta: Self::Metadata, subscriber: _, _param: u64) {
	let id = self.uid.fetch_add(1, atomic::Ordering::SeqCst);
	let sub_id = pubsub::SubscriptionId::Number(id as u64);
	let sink = subscriber.assign_id(sub_id.clone()).unwrap();
	self.active.write().unwrap().insert(sub_id, sink);
    }

Thank you

Tomasz Drwięga
@tomusdrw

https://github.com/paritytech/jsonrpc

https://github.com/tomusdrw/jsonrpc-demo

What is Blockchain?

  • Distributed data structure - to organize "transactions"/events
  • Consensus Algorithm - to decide who is allowed to modify that structure
  • Some crypto - to make it secure/tamperproof
  • Incentives - to make it run by itself

List of changes

List of changes

List of changes

Metadata

/ Previous state

Metadata

/ Previous state

Metadata

/ Previous state

Genesis State

Blockchain

Immutable data structure containing all the changes that were applied in any point in time

How does it work?

(Boot) Node 1

Node 2

New Node

Hey! Could you give me all your peers?

How does it work?

(Boot) Node 1

Node 2

New Node

Hey! Send me all them blocks, will ya?

Block 5

Block 4

Block 0

How does it work?

(Boot) Node 1

Node 2

New Node

Hey! I've got a transaction to include in block.

Block 5

Block 5

Block 5

transfer(N, B, 5)
sig(N)

How does it work?

(Boot) Node 1

Node 2

New Node

Block 5

Block 5

Block 5

transfer(N, B, 5)
sig(N)
transfer(N, B, 5)
sig(N)

Cool, I'm mining and will include the tx for a small fee.

How does it work?

(Boot) Node 1

Node 2

New Node

Block 6

Block 5

Block 5

transfer(N, B, 5)
sig(N)
transfer(N, B, 5)
sig(N)
Block 6

Managed to mine new block, here it is guys!

How does it work?

(Boot) Node 1

Node 2

New Node

Block 6

Block 6

Block 6

List of changes

List of changes

List of changes

Metadata

/ Previous state

Metadata

/ Previous state

Metadata

/ Previous state

Genesis State

Blockchain

Immutable data structure containing all the changes that were applied in any point in time

Blockchain

Hashes - prevent tampering (e.g. KECCAK256)

Signatures - authorize the actions (e.g. ECDSA)

Parent = hash(B0)
Timestamp = 150..000
Number = 1
Hash = hash(B1)
transfer(A, B, 5)
sig(A)
transfer(C, B, 1)
sig(C)
Parent = hash(B2)
Timestamp = 150..000
Number = 2
Hash = hash(B1)
transfer(B, A, 5)
sig(B)

Consensus Algorithm

Who is allowed to create new blocks?

sig(Authority1)
hash(B0)
hash(B1)
sig(Authority2)
hash(B2)
sig(Authority1)

Proof of Authority

We only accept blocks signed by a hardcoded list of authorities.

Blocks need to be signed in turns at most 1 block per 3 seconds.

Consensus Algorithm

Who is allowed to create new blocks?

Difficulty=2
Sol.=0b001..
SolvedBy=A
hash(B0)
hash(B1)
hash(B2)

Proof of Work

We only accept blocks with a solution to a puzzle.

The difficulty of the puzzle can be adjusted to have a stable rate of new blocks.

Difficulty=4
Sol.=0b00001..
SolvedBy=B
Difficulty=3
Sol.=0b0001..
SolvedBy=A

Why would you waste energy to create new blocks?

It's incentivised

=

You get a reward

Canonical Chain

What if two different blocks are produced with the same parent hash?

Which one should you choose?

Block 1

Block 2

Block 3

Block 3

Fork

Canonical Chain

We use "the longest" chain.

Ethereum re-organizes to a chain with the highest difficulty.

Block 1

Block 2

Block 3

Block 3

Block 4

Take away note: The latest state you see can sometimes be reverted - wait for confirmations.

Questions?

Blockchains allow for trustless transactions between multiple parties.

Made with Slides.com