$ whoami

• Tejas Manohar (@tejasmanohar)
• me@tejas.io
• Software Engineer at Segment

Agenda

• What's GRPC?
• Why GRPC?
• How GRPC?
  • ... with Node.js

What's GRPC?

What's GRPC?

• RPC protocol
  • Call func()
  • Ignore implementation
  • No REST, JSON, HTTP, WebSockets, etc.
• Powered by Protobuf
  • Binary => Performant
  • Typed => Self-documenting
• API client generator
  • Stop writing “wrappers”
  • Streaming, multiplexing, etc.

What GRPC is NOT?

• Monolithic web framework
  
  • Higher-level than Express
  • Lower-level than Rails
• Standard library
  • Not a standard, proprietary
  • High-level, opinionated

The Problem

Origin

analytics.track(‘userId’, ‘Clicked link’, {
  referralCode: ‘abc123’
})

var login

get('/token', function(err, token) {
  login.token = token
  if (login.token && login.key) login()
})

get('/key', function(err, key) {
  login.key = key;
  if (login.token && login.key) login()
})

function login() {
  post('/auth', [login.token, login.key], function(err, auth) {
    window.location.href = '/dashboard' + auth
  })
}

async.waterfall([
  function(callback) {
    async.parallel([
      function(callback) {
        get('/token', callback)
      },
      function(callback) {
        get('/key', callback)
      }
    ], callback)
  },
  function(callback) {
    post('/auth', { token: result[0], key: result[1] })
  }
], function() {
  window.location.href = '/dashboard?' + result  
})

var token = get('/token')
var key = get('/key')

Promise.all([token, key]).then(function(result) {
  return post('/auth') {
    token: result[0],
    key: result[1]
  })
}).then(function(result) {
  window.location.href = '/dashboard?' + result
})

Generators

• ES2015 feature
• Pause-able/iterable function
• Start, pause at yield without blocking outside scope, keep next() to iterate until generator is closed
• Work with resolved values
• Still non-blocking I/O

function* fibonacci(){
  var fn1 = 1;
  var fn2 = 1;
  while (true){  
    var current = fn2;
    fn2 = fn1;
    fn1 = fn1 + current;
    var reset = yield current;
    if (reset){
        fn1 = 1;
        fn2 = 1;
    }
  }
}

var sequence = fibonacci();
console.log(sequence.next().value);     // 1
console.log(sequence.next().value);     // 1
console.log(sequence.next().value);     // 2
console.log(sequence.next().value);     // 3
console.log(sequence.next().value);     // 5
console.log(sequence.next().value);     // 8
console.log(sequence.next().value);     // 13
console.log(sequence.next(true).value); // 1
console.log(sequence.next().value);     // 1
console.log(sequence.next().value);     // 2
console.log(sequence.next().value);     // 3

Generators w/ Co

• Co -> Generator based control flow goodness for nodejs and the browser
• Co 4.x.x returns Promises
• Stepping-stone to ES7 async functions

// from callbacks
function findFriends(user, callback) {
  Friend.find({ user: user }, function(err, friends) {
    if (err) {
      return callback(err)
    }
    callback(friends);
  });
}


// to promises
function findFriends(user) {
  return Friend.find({ user: user });
}

findFriends(user)
  .then(function(friends) {
    anotherAsyncOp(friends);
  })
  .catch(function(err) {
    someHandler(err);
  })


// to generators
function* findFriends(user) {
  return yield Friend.find({ user: user });
}

var x = yield findFriends(user);

yield says STOP

But... yield sounds...

Async operations

• Co allows non-blocking yet sequential code
• Filenames read synchronously but each file in dir read in parallel

var fs = require('co-fs');

app.use(function *(){
  var paths = yield fs.readdir('docs');

  var files = yield paths.map(function(path){
    return fs.readFile('docs/' + path, 'utf8');
  });

  this.type = 'markdown';
  this.body = files.join('');
});

Error-handling

Callbacks

• Remember if (err) cb(err)?
• Easy to miss errors
• Poor readability
• No true Error's thrown

Promises

• .catch()
• Have to adapt... doesn't feel sync
• Building long promise "chains" - improvement?

Generators

• try { ... } catch (e) { ... }
• Feels sync, like normal error-handling
• Eliminates the "necessary evil" in async programming

Decorators

Decorators

• Decorator accepts a function, wraps ("decorates") its call, and returns wrapper, which alters default behavior.
• What do middleware do?

/*
  App-
    1. Takes input
    2. Multiplies by 2
    3. Returns new value
*/

function app(number) {
  return number * 2;
}

app(10);

// Decorator
function decorate(multiplier) {
  return function passIn(fn) {
    /* Decorated fn ->
         1. manipulates input
         3. calls function
         4. manipulates output
    */
    return function decoratedFn(number) {
      number++;
      return multiplier * fn(number);
    };
  };
}

Using Decorators

• Decorators change your function
• Not a common design pattern in JS since we can contain the decoratee, but it's the basis of HTTP middleware
• Common in Python

decorate(2)(app)(10) // => 44
// (10 + 1) * 2 * 2 = 44

// expanded form ->
app(10) // => 20
app = decorate(2)(app)
app(10) // => 44

Middleware

Middleware

• Composed of decorator functions
• Mutates req and res objects
• Decorator -> Decorator -> Decorator -> ... chain

Express Middleware

• Mutating req, res
• Order matters
• Monkey-patching

app.use(function(req, res, next) {
  var writeHead = res.writeHead
    res.writeHead = function(status) {
      res.writeHead = writeHead
        if (status === 200) {
          res.header('Content-Disposition',
            'attachment; filename=' 
            + require('path').basename(req.url))
        }
        res.writeHead.apply(res, arguments)
    }
    next()
})

• Express uses Node's req/res objects
• res.end() prevents decoration w/o mutations
• This makes order of middleware matter
• No async code after response is written

function responseTime(options) {
  var opts = options || {}
  if (typeof options === 'number') {
    deprecate('number argument: use {digits: '
    + JSON.stringify(options) + '} instead')
    opts = { digits: options }
  }
  var fn = typeof opts !== 'function'
    ? createSetHeader(opts)
    : opts
  return function responseTime(req, res, next) {
    var startAt = process.hrtime()
    onHeaders(res, function onHeaders() {
      var diff = process.hrtime(startAt)
      var time = diff[0] * 1e3 + diff[1] * 1e-6
      fn(req, res, time)
    })
    next()
  }
}
function createSetHeader(options) {
  var digits = options.digits !== undefined
    ? options.digits
    : 3
  var header = options.header || 'X-Response-Time'
  var suffix = options.suffix !== undefined
    ? Boolean(options.suffix)
    : true
  return function setResponseHeader(req, res, time) {
    if (res.getHeader(header)) return
    var val = time.toFixed(digits)
    if (suffix) val += 'ms'
    res.setHeader(header, val)
  }
}

• After res.end(), async operations impossible
• If error, log it?
• Can't handle errors

var end = res.end;
res.end = function(data, encoding){
  res.end = end;
  if (!req.session) return res.end(data, encoding);
  req.session.resetMaxAge();

  if (resaveSession || isModified(req.session)) {
    debug('saving');
    return req.session.save(function(err){
      if (err) console.error(err.stack);
      debug('saved');
      res.end(data, encoding);
    });
  }

  res.end(data, encoding);
};

Koa Middleware

• Koa abstracts Node's native req/res objects
• Generators :)
• yield* waits on biz logic
• Bottom-up
• Truly a "stack"

function responseTime() {
  return function *responseTime(next){
    var start = Date.now();
    yield* next;
    var delta = Math.ceil(Date.now() - start);
    this.set('X-Response-Time', delta + 'ms');
  }
}

1. ​Create a date to track duration
2. yield control to the next middleware
3. Create another date to track response time
4. yield control to the next middleware
5. yield immediately since contentLength only works with responses
6. yield upstream to Koa's noop middleware
7. Ignore setting the body unless the path is "/"
8. Set the response to "Hello World"
9. Ignore setting Content-Length when no body is present
10. Set the field
11. Output log line
12. Set X-Response-Time header field before response
13. Hand off to Koa to handle the response

Composition

• Multiple middleware -> Single middleware
• Good for export / re-use

function *random(next) {
  if ('/random' == this.path) {
    this.body = Math.floor(Math.random()*10);
  } else {
    yield next;
  }
};

function *backwards(next) {
  if ('/backwards' == this.path) {
    this.body = 'sdrawkcab';
  } else {
    yield next;
  }
}

function *pi(next) {
  if ('/pi' == this.path) {
    this.body = String(Math.PI);
  } else {
    yield next;
  }
}

function *all(next) {
  yield random.call(this,
        backwards.call(this,
        pi.call(this, next)));
}

app.use(all);

Compose

• koa-compose
• Koa internally uses this to create and dispatch the middleware stack

app.use(compose([random, backwards, pi]))

Context

req, res, ...???

A Koa application is an object containing an array of middleware generator functions which are composed and executed in a stack-like manner upon request

Context

• With each request, new context is created
• Though Koa wraps Node's req/res objects, it still gives you native access
• Node's req/res objects are 100% compatible w/ Express

app.use(function *(){
  this.body = 'Hello World';
});

app.use(function *(){
  this; // context
  this.request; // koa request
  this.response; // koa response
  this.req; // node.js request
  this.res; // node.js response
});

Routing

Routing in Koa

• Koa comes with no middleware
  • And, since everything is middleware...
    • This means no router!
• This doesn't mean you can't use a router!
• koa-route is most popular, implemented in ~30 SLOC

Route

• Hello World app in Express, Koa, and Koa w/ koa-route
• But... this is all basic, not the real power of Koa!

// express
var express = require('express');
var app = express();

app.get('/', function(req, res){
  res.send('hello world');
});

// koa
var koa = require('koa');
var app = koa();

app.use(function *() {
  if (url === '/') this.body = 'Hello World';
});

// koa-route
var koa = require('koa');
var app = koa();
var route = require('koa-route');

app.use(route.get('/', function *() {}
    this.body = 'Hello World';
));

Debugging

DEBUG

• Uses debug module for conditional logging
• Pass DEBUG=koa* for list of middleware used, etc.
• Most Koa middleware does same

$ DEBUG=koa* node examples/simple
  koa:application use responseTime +0ms
  koa:application use logger +4ms
  koa:application use contentLength +0ms
  koa:application use notfound +0ms
  koa:application use response +0ms
  koa:application listen +0ms

Middleware

• JS doesn't allow defining fn names at runtime
• You can also set a middleware's name as ._name
• Good for imported/3rd-party middleware

var path = require('path');
var static = require('koa-static');

var publicFiles = static(path.join(__dirname, 'public'));
publicFiles._name = 'static /public';

app.use(publicFiles);

ES6

Arrow Functions

var self = this

• this depends on "context"
• Only option <= ES4

function Person() {
  var self = this;

  self.age = 0;

  setInterval(function growUp() {
    self.age++;
  }, 1000);
}

.bind(this)

• ES5 introduced Function#bind
• function(){}.bind(this)

function Person() {
  setInterval(function growUp() {
    this.age++;
  }.bind(this), 1000);
}

() => {}

• Lexical binding
• Sounds familiar to Coffeescript users
• Short expressions

function Person(){
  this.age = 0;

  setInterval(() => {
    this.age++;
  }, 1000);
}

var p = new Person();

Other cases

• Short expressions
• Implicit return without {}

(param1, param2, paramN) => { statements }
(param1, param2, paramN) => expression

singleParam => { statements }
singleParam => expression

() => { statements }

params => ({foo: bar})

Classes

• Syntactical sugar over JavaScript's existing prototype-based inheritance
• NOT introducing new OO inheritance model, only syntax

class Animal { 
  constructor(name) {
    this.name = name;
  }
  
  speak() {
    console.log(this.name + ' makes a noise.');
  }
}

class Dog extends Animal {
  speak() {
    console.log(this.name + ' barks.');
  }
}

Object Literals

• Set the prototype at construction
• Shorthand for foo: foo and defining functions

var obj = {
    // __proto__
    __proto__: theProtoObj,
    // Does not set internal prototype
    '__proto__': somethingElse,
    // Shorthand for ‘handler: handler’
    handler,
    // Methods
    toString() {
     // Super calls
     return "d " + super.toString();
    },
    // Computed (dynamic) property names
    [ "prop_" + (() => 42)() ]: 42
};

Destructuring

• Basic pattern matching
• In arrays, by position
• In objects, by key
• Fail-soft just like object lookup obj.x (undefined)

// list matching
var [a, , b] = [1,2,3];

// object matching
var { op: a, lhs: { op: b }, rhs: c }
       = getASTNode()

// object matching shorthand
// binds `op`, `lhs` and `rhs` in scope
var {op, lhs, rhs} = getASTNode()

// Can be used in parameter position
function g({name: x}) {
  console.log(x);
}
g({name: 5})

// Fail-soft destructuring
var [a] = [];
a === undefined;

// Fail-soft destructuring with defaults
var [a = 1] = [];
a === 1;

Template Strings

• a.k.a. string interpolation
• Avoids ugly concatenation
• Improves readability
• Full multi-line support
• Unescaped "/' characters

// Basic literal string creation
`This is a pretty little template string.`

// Multiline strings
`In ES5 this is
 not legal.`

// Interpolate variable bindings
var name = "Bob", time = "today";
`Hello ${name}, how are you ${time}?`

// Unescaped template strings
String.raw`In ES5 "\n" is a line-feed.`

Default

• Default function parameters
• Avoid undefined

function f(x, y=12) {
  return x + y;
}
f(3) == 15 // => true

Rest

• Last n parameters join to become an array
• Avoid hard limits on function parameters

function f(x, ...y) {
  // y is an Array
  return x * y.length;
}
f(3, "hello", true) == 6

Spread

• Array -> Arguments
• Under the hood, it's a form of pattern matching / destructuring

function f(x, y, z) {
  return x + y + z;
}
f(...[1,2,3]) == 6 // => true

Block-scope

• Block-scoped binding constructs
• let is the new var
• const is single-assignment
  • NOT immutable!
  • Constant describes references, not values.
• Block-scope declarations aren't hosted by spec
  • But in Babel... they are transpiled to var

function f() {
  {
    let x;
    {
      // okay, block scoped name
      const x = "sneaky";
      // error, const
      x = "foo";

      const obj = { hi: 2 };
      // valid
      obj.hi = 3;
    }
    // okay, declared with `let`
    x = "bar";
    // error, already declared in block
    let x = "inner";
  }
}

Iterators

• for... of loop
• Works for all Objects (Arrays, too)
• Loops are still loops

for (var n of fibonacci) {
  // truncate the sequence at 1000
  if (n > 1000)
    break;
  console.log(n);
}

Unicode

• Unicode literal form in strings and RegExp
• Good for internationalization

// same as ES5.1
"𠮷".length == 2

// new RegExp behaviour, opt-in ‘u’
"𠮷".match(/./u)[0].length == 2

// new form
"\u{20BB7}" == "𠮷" == "\uD842\uDFB7"

// new String ops
"𠮷".codePointAt(0) == 0x20BB7

// for-of iterates code points
for(var c of "𠮷") {
  console.log(c);
}

Modules

• Not a replacement
• 2nd-class and merely a wrapper over require()
• Implicitly async
• Statically analyzed
  • But not yet optimized
• No new module loader in ES6, still up to environment

// lib/math.js
export function sum(x, y) {
  return x + y;
}
export var pi = 3.141593;

// app.js
import * as math from "lib/math";
alert("2π = " + math.sum(math.pi, math.pi));

// otherApp.js
import {sum, pi} from "lib/math";
alert("2π = " + sum(pi, pi));

Maps/Sets

• Real data structures coming to JS
• Maps are great for dynamic objects due to explicit getter/setter
  • i.e. Flux Stores

// Sets
var s = new Set();
s.add("hello")
  .add("goodbye")
  .add("hello");
s.size === 2;
s.has("hello") === true;

// Maps
var m = new Map();
m.set("hello", 42);
m.set(s, 34);
m.get(s) == 34;

// Weak Maps
var wm = new WeakMap();
wm.set(s, { extra: 42 });
wm.size === undefined

// Weak Sets
var ws = new WeakSet();
ws.add({ data: 42 });

Symbols

• Enable access control for object state
• Props can be keyed by string or symbol
• New primitive type

(function() {

  // module scoped symbol
  var key = Symbol("key");

  function MyClass(privateData) {
    this[key] = privateData;
  }

  MyClass.prototype = {
    doStuff: function() {
      ... this[key] ...
    }
  };

  typeof key === "symbol"
})();

var c = new MyClass("hello")
c["key"] === undefined

Subclasses

• Built-ins (Date, Array, etc.) can be sub-classed
• Useful for extending built-ins- i.e. Moment

// User code of Array subclass
class MyArray extends Array {
    constructor(...args) { super(...args); }
}

var arr = new MyArray();
arr[1] = 12;
arr.length == 2

Math, Number, String, Object

Number.EPSILON
Number.isInteger(Infinity) // false
Number.isNaN("NaN") // false

Math.acosh(3) // 1.762747174039086
Math.hypot(3, 4) // 5
Math.imul(Math.pow(2, 32) - 1, Math.pow(2, 32) - 2) // 2

0b111110111 === 503 // true
0o767 === 503 // true

"abcde".includes("cd") // true
"abc".repeat(3) // "abcabcabc"

Array.from(document.querySelectorAll("*")) // Returns a real Array
Array.of(1, 2, 3)
[0, 0, 0].fill(7, 1) // [0,7,7]
[1,2,3].findIndex(x => x == 2) // 1
["a", "b", "c"].entries() // iterator [0, "a"], [1,"b"], [2,"c"]
["a", "b", "c"].keys() // iterator 0, 1, 2
["a", "b", "c"].values() // iterator "a", "b", "c"

Object.assign(Point, { origin: new Point(0,0) })

Tail calls

• Calls in tail-position are guaranteed to not grow the stack unboundedly
• Avoid stack overflows
• Safer recursion

function factorial(n, acc = 1) {
    "use strict";
    if (n <= 1) return acc;
    return factorial(n - 1, n * acc);
}

factorial(100000)

Workshop Time!