Generators & co

Who is this presentation made for ?

For people struggling with writing asynchronous JavaScript code
For people wanting to learn something different

The issue

In a JavaScript function, code is asynchronous.

hard to get things done sequentially
or in parallel

In a JavaScript generator controlled by "co"

the same code can be written as if it were synchronous
but still be executed in parallel with other code

The answer

Plan

Generators: in a few words
Generators: syntax
Generators: control layer
"co" as a control layer
Let's code ;)

Generators: in a few words

Support

Supported from node v0.11

Main facts

A generator is code which stops when yielding values
It generates iterators of what it yields
Iterating resumes the code

now, let's start from the beginning

Generators: syntax

Generator declaration

// in a scope
function* gen() { /* ... */ }

// in an object
const a = {
  gen: function* () { /* ... */ }
}

// in a class
class A {
  *gen() { /* ... */ }
}

Generator's body

function* gen1() {
    yield 1;
    yield 2;
    yield 3;
}

function* gen2() {
    yield 0;
    yield* gen1(); // delegation
    yield 4;
}

// `Array.from` handles iterators, it iterates till the end
const iter = gen2();
console.log(Array.from(iter)); // [0, 1, 2, 3, 4]

Iterations are code







const a = [1, 2, 3]; // [1, 2, 3]

function* gen() {
    yield 1;
    yield 2;
    yield 3;
}

const a = Array.from(gen()); // [1, 2, 3]

Declarative

Programmatic

Yielded objects

The control layer gets yielded values wrapped into an object of type:
{value: any, done: boolean}

"yield A" -> {value: A, done: false}

"return B" -> {value: B, done: true}

Control the iterator with next()

// iteration code
function* gen() {
    yield 1;
    yield 2;
    yield 3;
}

// control code
const iter = gen();
console.log(iter.next()); // {value: 1, done: false}
console.log(iter.next()); // {value: 2, done: false}
console.log(iter.next()); // {value: 3, done: false}
console.log(iter.next()); // {value: undefined, done: true}

Communication summary

GENERATOR	DIR.	CONTROL
yield message	→	{value: message, done: false} = iter.next(...)

Generators: control layer

Control is about resuming the iterator

Generates an iterator "iter = gen()"
It's initially stopped
You start it with "iter.next()"
It stops with "yield A"
You resume it with "iter.next(B)"
It continues as if "yield A" was replaced by "B";

// resume it once, it's already done
console.log(iter.next(B)); // -> {value: B, done: true}

// generate the iterator
var iter = gen();

// start it
console.log(iter.next()); // -> {value: A, done: false}

function* gen() {
    return yield A;
}

Examples of control layers

One generator, different ways to control the iterations.

// iteration code
function* genArgs(...instantiationArgs) {
    while (instantiationArgs.length) { console.log(yield instantiationArgs.shift()); }
}

the considered iterator: iterates over the generator call arguments

CONTROL LAYERS: iterate

One generator, different ways to control the iterations.

// iteration code
function* genArgs(...instantiationArgs) {
    while (instantiationArgs.length) { console.log(yield instantiationArgs.shift()); }
}

drop the emitted value, resume with undefined

// instantiate iterator
var iter = genArgs(1, 2, 3);

// control: resume with undefined, repeated 4 times
console.log(iter.next()); // RESUME WITH UNDEFINED

// {value: 1, done: false}
// undefined

// {value: 3, done: false}
// undefined

// {value: 2, done: false}
// undefined

// {done: true}

CONTROL LAYERS: synchronous WATERFALL

One generator, different ways to control the iterations.

// iteration code
function* genArgs(...instantiationArgs) {
    while (instantiationArgs.length) { console.log(yield instantiationArgs.shift()); }
}

resume with the emitted value

// instantiate iterator
var iter = genArgs(1, 2, 3);

// retain last emit
var emitted = {value: undefined, done: false};

// control: resume with previous emitted value, repeated 4 times
console.log(emitted = iter.next(emitted.value)); // RESUME WITH THE EMITTED VALUE

// {value: 1, done: false}
// 1
// {value: 2, done: false}
// 2
// {value: 3, done: false}
// 3
// {done: true}

CONTROL LAYERS: asynchronous WATERFALL

One generator, different ways to control the iterations.

// iteration code
function* genArgs(...instantiationArgs) {
    while (instantiationArgs.length) { console.log(yield instantiationArgs.shift()); }
}

resume with the resolve value of the emitted promise

// control code
var iter = genArgs(Promise.resolve(1), Promise.resolve(2), Promise.resolve(3));

var chain = Promise.resolve();

// promise chain control, repeated
chain = chain.then((value) => {
    const emitted = iter.next(value); // RESUME WITH THE RESOLVE VALUE
    console.log(emitted);
    return emitted.value;
});

// {value: Promise<1>, done: false}
// 1
// {value: Promise<2>, done: false}
// 2
// {value: Promise<3>, done: false}
// 3
// {done: true}

Communication summary

GENERATOR	DIR.	CONTROL
yield message	→	{value: message, done: false} = iter.next(...)
message = yield	←	iter.next(message)

Catching errors

For simplicity purpose, we voluntarily don't pay attention to error handling, so you can skip this part
FYI:
- Iterator and control code can catch their own execution errors
- Control code can for example catch yielded promise rejection and resume the iterator with the error

GENERATOR	DIR.	CONTROL
try { yield; } catch(error) {}	→	iter.throw(error)
throw error	←	try { iter.next(...); } catch(error) {}

Co

"co" on NPM and Github

To date 2016-09-25

"Generator based control flow goodness for nodejs and the browser, using promises, letting you write non-blocking code in a nice-ish way"

"co" like "coroutine"

warning: it's not exactly coroutines
generators are semi-coroutines (or asymmetric coroutines)
"co" is an implementation of the control layer

Link with the last example

// iteration code
function* genArgs(...instantiationArgs) {
    while (instantiationArgs.length) { console.log(yield instantiationArgs.pop()); }
}


// control code
var iter = genArgs(Promise.resolve(1), Promise.resolve(2), Promise.resolve(3));
var chain = Promise.resolve();

// promise chain control, repeated
chain = chain.then((value) => iter.next(value).value);

"co" implements this control layer (not exactly)

You will only implement this part

Control layer

It handles async yielded objects and resolves them before resuming with the resolve value, i.e.

It actually handles the following types:

promises
thunks (functions) [deprecated]
array (parallel execution)
objects (parallel execution)
generators (delegation)
generator functions (delegation)

Example

const customProcess = () => {
  const authPromise = authenticate();
  authPromise.then(storeTokens);

  return authPromise
    .then(() => getProfile('me'))
    .then(({accountId}) => Promise.all([
      getRestaurant(accountId),
      getReservations(accountId)
    ]))
    .then(([restaurant, reservations]) => ({
      restaurant,
      reservations
    }));
}

const customProcess = co.wrap(function* () {
  const auth = yield authenticate();
  storeTokens(auth);



  const {accountId} = yield getProfile('me');
    


  return yield {
    restaurant: getRestaurant(accountId),
    reservations: getReservations(accountId),
  };
});

Promises only

Wrapped with "co"

And that's it !

You write synchronous code, yielding only when it's asynchronous.

The control layer will resume your code when ready ! (and do some more clever things)

And what about async/await ES7 feature ?

Consider it's the same but:

as a language feature (ES7)
not a library

Let's code

Prepare

Ensure you're using a recent node version

// execute on file change
while inotifywait -q <filename>; do node $_; done

To execute your code automatically, you can use:

from package "inotify-tools"

for mac users, you can user "fswatch"

Start from https://github.com/atondelier/coding-dojo-co

Synchronous pause

Implement a "wait" function which can be used to pause an iterator like this:

"use strict";
const co = require('co');

const wait = (ms) => {
  /* implement wait body */
};

co(function* (){
    console.log('before');
    yield wait(1000);
    console.log('after');
});

Sized Batch

Implement the generator which will resolve all the requests in sized batches

"use strict";
const co = require('co').default;
const _ = require('lodash');

// do request which resolves between 1 and 2 seconds after
const request = (path) => new Promise((resolve) => {
    console.log('start ' + path);
    setTimeout(
        () => {
            console.log('finish ' + path);
            resolve({path, content: 'content for path ' + path })
        },
        (1 + Math.random()) * 1000
    );
});

co(function* (){

    const paths = _.range(0, 10).map((n) => 'path/to/file' + n);
    const contents = [];

    /* implement sized batching here */

    return contents;                

}).then(console.log, console.error);

Sized parallel

Implement the generator which will process requests in parallel
and return the contents without changing the order, with a maximum of N requests in parallel

"use strict";
const co = require('co').default;
const _ = require('lodash');

// do request which resolves between 1 and 2 seconds after
const request = (path) => new Promise((resolve) => {
    console.log('start ' + path);
    setTimeout(() => {
        console.log('finish ' + path);
        resolve({path, content: 'content for path ' + path })
    }, (1 + Math.random()) * 1000);
});

co(function* (){

    const paths = _.range(0, 10).map((n) => 'path/to/file' + n);
    const contents = [];

    /* implement sized parallel process here */

    return contents;

}).then(console.log, console.error);

Sized parallel

BONUS: extract this behavior in an external generator