5 Architectures of
Asynchronous JavaScript

Tomasz Ducin

@tomasz_ducin

5 Architectures of Asynchronous JavaScript

Tomasz Ducin

7th June 2019, Amsterdam

5 Architectures of Asynchronous JavaScript

5 Architectures of Asynchronous JavaScript

Tomasz Ducin

11th October 2017, Warsaw

#38

Tomasz Ducin

10th October 2017, Warsaw

#216

5 Architectures of Asynchronous JavaScript

5 Architectures of Asynchronous JavaScript

5 Architectures of Asynchronous JavaScript

Tomasz Ducin

Tomasz Ducin

Tomasz Ducin

13th July 2017, London

FullStack

FullStack

5 Architectures of Asynchronous JavaScript

conference on JavaScript, Node & Internet of Things

conference on JavaScript, Node & Internet of Things

Tomasz Ducin

13th July 2017, London

5 Architectures of Asynchronous JavaScript

Tomasz Ducin

22nd June 2017, Kraków

5 Architectures of
Asynchronous JavaScript

Tomasz Ducin

3rd April 2017, Warsaw

5 Architectures of
Asynchronous JavaScript

Tomasz Ducin

30th March 2017, Warsaw

5 Architectures of
Asynchronous JavaScript

Tomasz Ducin

developer & architect

independent consultant

trainer @ Bottega IT Minds

@tomasz_ducin

ducin

ducin.it

functions

sync vs async

promises

async await

coroutines

CSP

generators

callbacks

events

reactive streams

run to completion

event loop

sync vs async

blocking vs non-blocking

code is linear; execution:

linear vs non-linear

learn by metaphors

Concurrent

Parallel

no thread synchronization

no deadlocks

Concurrency

cooperative

vs

preemptive

setInterval & setTimeout

UX/performance issues

function-first

  • scopes

  • closures

  • context, binding

  • legacy (hoisting, IIFEs)

  • FP, side effects, purity

  • etc.

list.forEach(function(item){
  ...
});

synchronous

asynchronous

setTimeout(function(){
  ...
}, delay);
var list = [1,2,3];
console.log("before");
list.forEach(function(item){
  console.log(item);
});
console.log("after");
console.log("before");
setTimeout(function(){
  console.log("inside");
}, 0);
console.log("after");

callbacks

asynchronous

setTimeout(function(){
  ...
}, delay);
......................
...........function(){
  console.log("inside");
}.....
.....................

callbacks

event loop

message queue

one thing at a time

(single thread) 

(function() {
    console.log(1); 
    setTimeout(function(){console.log(2)}, 1000); 
    setTimeout(function(){console.log(3)}, 0); 
    console.log(4);
})();
console.log(5);

run to completion

(function() {
    console.log(1); 
    setTimeout(.........................., 1000); 
    setTimeout(.........................., 0); 
    console.log(4);
})();
console.log(5);
.............
    ............... 
    ............................................. 
    ...........function(){console.log(3)}..... 
    ............... 
.....
...............
.............
    ............... 
    ...........function(){console.log(2)}........ 
    .......................................... 
    ............... 
.....
...............

Registering callbacks

$(document).ready(function() {
  $('.pull-me').click(function() {
    $('.panel').slideToggle('slow');
  });
});

& run to completion

var customers;

// usually takes 0,5s
ajax.get('customers', {
  success: function(data){
    customers = data;
  }
});

assumptions about
async order 

var customers, products;

// usually takes 0,5s
ajax.get('customers', {
  success: function(data){
    customers = data;
  }
});

// usually takes 2s
ajax.get('products', {
  success: function(data){
    doSomething(customers, data);
  }
});
connection.query('CREATE DATABASE IF NOT EXISTS test', function (err) {
    if (err) throw err;
    connection.query('USE test', function (err) {
        if (err) throw err;
        connection.query('CREATE TABLE IF NOT EXISTS users('
            + 'id INT NOT NULL AUTO_INCREMENT,'
            + 'PRIMARY KEY(id),'
            + 'name VARCHAR(30)'
            +  ')', function (err) {

            });
    });
});
connection.query('CREATE DATABASE IF NOT EXISTS test', function (err) {











});
connection.query('CREATE DATABASE IF NOT EXISTS test', function (err) {
    if (err) throw err;
    connection.query('USE test', function (err) {








    });
});
connection.query('CREATE DATABASE IF NOT EXISTS test', function (err) {
    if (err) throw err;
    connection.query('USE test', function (err) {
        if (err) throw err;
        connection.query('CREATE TABLE IF NOT EXISTS users('
            + 'id INT NOT NULL AUTO_INCREMENT,'
            + 'PRIMARY KEY(id),'
            + 'name VARCHAR(30)'
            +  ')', function (err) {
                if (err) throw err;
            });
    });
});

callback hell

flying V

pyramid of doom

fs.readdir(source, function (err, files) {
  if (err) {
    console.log('Error finding files: ' + err)
  } else {
    files.forEach(function (filename, fileIndex) {
      console.log(filename)
      gm(source + filename).size(function (err, values) {
        if (err) {
          console.log('Error identifying file size: ' + err)
        } else {
          console.log(filename + ' : ' + values)
          aspect = (values.width / values.height)
          widths.forEach(function (width, widthIndex) {
            height = Math.round(width / aspect)
            console.log('resizing ' + filename + 'to ' + height + 'x' + height)
            this.resize(width, height).write(dest + 'w' + width + '_' + filename, function(err) {
              if (err) console.log('Error writing file: ' + err)
            })
          }.bind(this))
        }
      })
    })
  }
})

Callback hell

a Promise of
a single operation
to be completed in future 

fetchData()
.then(callbackFn)
.catch(errorHandlerFn)
myCustomOperation()
.then(callbackFn)
.catch(errorHandlerFn)

Promise - states

pending

fulfilled

rejected

initial state,
not settled yet

operation has failed

operation has completed successfully

settled

promise limitations

  • single item

  • one-time item

  • greedy

  • not cancellable

  • values unavailable outside the chain

  • only previous step value available 


  return asyncOp1();
function promiseChain(){
  return asyncOp1()
    .then(asyncOp2)
    .then(asyncOp3)
    .then(asyncOp4);
}

  return asyncOp1()
    .then(asyncOp2);

  return asyncOp1()
    .then(asyncOp2)
    .then(asyncOp3);

  return asyncOp1()
    .then(asyncOp2)
    .then(asyncOp3)
    .then(asyncOp4);

function sequential(){
  return asyncOp1()
    .then(asyncOp2)
    .then(asyncOp3)
    .then(asyncOp4);
}

Sequential Processing

op1
op2
op3
op4
sequential()
  .then(anotherAsync)
another
function parallel(){
  var p1 = asyncOp1();
  var p2 = asyncOp2();
  var p3 = asyncOp3();
  var p4 = asyncOp4();
  return Promise.all(
    [p1, p2, p3, p4]);
}

Concurrent Processing

op1
another
.all
parallel()
  .then(anotherAsync)
op2
op3
op4
Promise.all([p1, p2, p3, p4])
  .then( ([v1, v2, v3, v4]) => {...})
  .catch( reason => {...})

Promise Aggregates

Promise.race([p1, p2, p3, p4])
  .then( value => {...})
  .catch( reason => {...})
Promise.any([p1, p2, p3, p4])
  .then( value => {...})
  .catch( ([r1, r2, r3, r4]) => {...})

all
race
any
Promise.some([p1, p2, p3, p4], 2)
  .then( (v1, v2) => {...})
  .catch( ([r1, r2, r3]) => {...})

some
function zeroOneTwo(){
  return [0, 1, 2];
}
function* zeroOneTwo(){
  yield 0;
  yield 1;
  yield 2;
}

lazy

greedy

list

generator

synchronous

synchronous

for (var i of zeroOneTwo()) {
  console.log(i);
} // 1 2 3
function* generator(){
  console.log(1, "inside");
  yield "A";
  console.log(2, "inside");
  yield "B";
}

var iterator = generator();
console.log(1, "outside");
iterator.next();
console.log(2, "outside");
iterator.next();
console.log(3, "outside");

synchronous

Generators

vs run to completion

1: outside
1: inside
2: outside
2: inside
3: outside
function* generate(){
  console.log(1, "inside");
  let recv = yield "A";
  console.log(2, "inside", recv);
  yield "B";
}

var iter = generate();
console.log(1, "outside");
let item = iter.next();
console.log(2, "outside", item.value);
iter.next('hey!');
console.log(3, "outside");

synchronous

Generators

vs run to completion

1: outside
1: inside
2: outside, A
2: inside, hey!
3: outside

= coroutines

promises + generators

+ 1 tiny wrapper

  • start immediately
  • suspend on promise
     
  • resume when settled
  • promise pending...
  • resolve/ reject calls next

writing asynchronous code in synchronous manner 

function* sequential(){
  var v1 = yield asyncOp1();
  var v2 = yield asyncOp2(v1);
  var v3 = yield asyncOp3(v2);
  return asyncOp4(v3);
}
let asyncSequential
  = async(sequential);

Sequential Processing

asyncSequential()
  .then(anotherAsync)
op1
op2
op3
op4
another
function* parallel(){
  var p1 = asyncOp1();
  var p2 = asyncOp2();
  var p3 = asyncOp3();
  var p4 = asyncOp4();
  return yield p1 + yield p2
    + yield p3 + yield p4;
}
let asyncParallel
  = async(parallel)
asyncParallel()
  .then(anotherAsync)
op1
another
.all
op2
op3
op4

Concurrent Processing

function async(makeGenerator){
  return function () {
    var generator = makeGenerator.apply(this, arguments);

    function handle(result){
      // result => { done: [Boolean], value: [Object] }
      if (result.done) return Promise.resolve(result.value);
      return Promise.resolve(result.value).then(function (res){
        return handle(generator.next(res));
      }, function (err){
        return handle(generator.throw(err));
      });
    }

    try {
      return handle(generator.next());
    } catch (ex) {
      return Promise.reject(ex);
    }
  }
}

a couple of lines that
do the right thing...

ES2017 / ES8

async function sequential(){
  var v1 = await asyncOp1();
  var v2 = await asyncOp2(v1);
  var v3 = await asyncOp3(v2);
  return asyncOp4(v3);
}

Sequential Processing

sequential()
  .then(anotherAsync)
op1
op2
op3
op4
another
async function parallel(){
  var p1 = asyncOp1();
  var p2 = asyncOp2();
  var p3 = asyncOp3();
  var p4 = asyncOp4();
  return await p1 + await p2
    + await p3 + await p4;
}
parallel()
  .then(anotherAsync)
op1
another
.all
op2
op3
op4

Concurrent Processing

async/await usecase 

model {
  beginTransfer(): Promise<TransferID> {
    return HTTP.post('/transfers');
  }











}
model {
  beginTransfer(): Promise<TransferID> {
    return HTTP.post('/transfers');
  }

  setTransferDetails(
    transferID: TransferID,
    details: TransferDetails
  ): Promise<void> {
    return HTTP.post(`/transfers/${transferID}`, details);
  }




}
model {
  beginTransfer(): Promise<TransferID> {
    return HTTP.post('/transfers');
  }

  setTransferDetails(
    transferID: TransferID,
    details: TransferDetails
  ): Promise<void> {
    return HTTP.post(`/transfers/${transferID}`, details);
  }

  confirmTransfer(transferID: TransferID): Promise<?> {
    return HTTP.post(`/transfers/${transferID}`, {action: "confirm"});
  }
}
model {














}
async function scheduleTransfer(component){
  let transferID = await model.beginTransfer();









}
async function scheduleTransfer(component){
  let transferID = await model.beginTransfer();
  // waiting for user to input data
  await component.completedStep1();
  await model.setTransferDetails(transferID, details);
  // waiting for user confirmation
  await component.completedStep2();
  await model.confirmTransfer(transferID);
  // waiting for user closing popup
  await component.completedStep3();
  return transferID;
}

async function scheduleTransfer(component){
  let transferID = await model.beginTransfer();
  // waiting for user to input data
  await component.completedStep1();







}

async function scheduleTransfer(component){
  let transferID = await model.beginTransfer();
  // waiting for user to input data
  await component.completedStep1();
  await model.setTransferDetails(transferID, details);






}

async function scheduleTransfer(component){
  let transferID = await model.beginTransfer();
  // waiting for user to input data
  await component.completedStep1();
  await model.setTransferDetails(transferID, details);
  // waiting for user confirmation
  await component.completedStep2();




}
async function scheduleTransfer(component){
  let transferID = await model.beginTransfer();
  // waiting for user to input data
  await component.completedStep1();
  await model.setTransferDetails(transferID, details);
  // waiting for user confirmation
  await component.completedStep2();
  await model.confirmTransfer(transferID);



}

async function scheduleTransfer(component){
  let transferID = await model.beginTransfer();
  // waiting for user to input data
  await component.completedStep1();
  await model.setTransferDetails(transferID, details);
  // waiting for user confirmation
  await component.completedStep2();
  await model.confirmTransfer(transferID);
  // waiting for user closing popup
  await component.completedStep3();

}

function scheduleTransfer(component){
  return model.beginTransfer()
  .then(transferID => {
    // waiting for user to input data
    return component.completedStep1()
    .then(() => model.setTransferDetails(transferID, details))
    // waiting for user confirmation
    .then(() => component.completedStep2())
    .then(() => model.confirmTransfer(transferID))
    .then(() => component.completedStep3())
    .then(() => transferID);
  })
}
  • many nested function calls
  • callback hell again

Pipeline scope

function promiseChain(){
  return asyncOp1()
    .then(asyncOp2)
    .then(asyncOp3)
    .then(asyncOp4 ???);
}

using data from previous steps

async function sequential(){
  var v1 = await asyncOp1();
  var v2 = await asyncOp2(v1);
  var v3 = await asyncOp3(v2);
  return asyncOp4(v1, v2, v3);
}

No nested functions

same as in generators

async function renderChapters(urls) {
  urls.map(getJSON)
    .forEach(p => addToPage((await p).html));
}
async function renderChapters(urls) {
  urls.map(getJSON)
    .forEach(async p => addToPage((await p).html));
}
                                Syntax Error
                            
                                parallel

Async Arrow Functions

(async x => x ** 2);
(async x => { return x ** 2; });
(async (x, y) => x ** y);
(async (x, y) => { return x ** y; });

instead of Promise.resolve

const square = (async x => x ** 2);
square(5) // same as Promise.resolve(25)

square(5).then(console.log)
// output: 25

Async Iteration

stage 3 proposal 

for await (const line of readLines(filePath)) {
  console.log(line);
}
syncIterator.next()
// -> { value: ..., done: ... }
asyncIterator.next()
// -> Promise resolving with
//    { value: ..., done: ... }
.then(({ value, done }) => /* ... */);

lazy and asynchronous

Top-level await

// all awaited functions return a promise
// all awaited functions return a promise

await delay(2000);
// all awaited functions return a promise

await delay(2000);

// webservices
const flowers = await fetch('flowers.jpg');
// all awaited functions return a promise

await delay(2000);

// webservices
const flowers = await fetch('flowers.jpg');

// I/O operations
let content = '...';
let fileName = 'filename.json';
await writeFile(fileName, content);
await doSomeProcessing(fileName);

Top-level await is a footgun

Rich Harris, creator of Rollup

var data = await something();
// import gets blocked
import { m } from 'module';

m.run();
if (!Array.prototype.myMethod){
  ...
  await something();
}
var data = await something();
export default data;

module.js

Top-level await

consumer.js

Communicating Sequential Processes

C
S
P 

import {go, chan, take, put} from 'js-csp';

let chA = chan();
let chB = chan();
js-csp
yield put
yield take
chan
go
csp.go(function* () {
  const rec1 = yield csp.take(chA);
  console.log('A > RECEIVED:', rec1);

  const sent2 = 'cat';
  console.log('A > SENDING:', sent2);
  yield csp.put(chB, sent2);

  const rec3 = yield csp.take(chA);
  console.log('A > RECEIVED:', rec3);
});
csp.go(function* () {
  const sent1 = 'dog';
  console.log('B > SENDING:', sent1);
  yield csp.put(chA, sent1);

  const rec2 = yield csp.take(chB);
  console.log('B > RECEIVED:', rec2);

  const sent3 = 'WAT!';
  console.log('B > SENDING:', sent3);
  yield csp.put(chA, sent3);
});
=> B > SENDING: dog
=> A > RECEIVED: dog
=> A > SENDING: cat
=> B > RECEIVED: cat
=> B > SENDING: WAT!
=> A > RECEIVED: WAT!

process B

process A

csp.go(function* () {









});
csp.go(function* () {










});

process B

process A

csp.go(function* () {
  const rec1 = yield csp.take(chA);








});
csp.go(function* () {
  const sent1 = 'dog';
  console.log('B > SENDING:', sent1);








});
=> B > SENDING: dog
=> B > SENDING: dog
=> A > RECEIVED: dog
csp.go(function* () {
  const rec1 = yield csp.take(chA);
  console.log('A > RECEIVED:', rec1);







});
csp.go(function* () {
  const rec1 = yield csp.take(chA);
  console.log('A > RECEIVED:', rec1);

  const sent2 = 'cat';
  console.log('A > SENDING:', sent2);




});
csp.go(function* () {
  const sent1 = 'dog';
  console.log('B > SENDING:', sent1);
  yield csp.put(chA, sent1);

  const rec2 = yield csp.take(chB);





});
=> B > SENDING: dog
=> A > RECEIVED: dog
=> A > SENDING: cat
csp.go(function* () {
  const rec1 = yield csp.take(chA);
  console.log('A > RECEIVED:', rec1);

  const sent2 = 'cat';
  console.log('A > SENDING:', sent2);
  yield csp.put(chB, sent2);

  const rec3 = yield csp.take(chA);

});
csp.go(function* () {
  const sent1 = 'dog';
  console.log('B > SENDING:', sent1);
  yield csp.put(chA, sent1);

  const rec2 = yield csp.take(chB);
  console.log('B > RECEIVED:', rec2);




});
=> B > SENDING: dog
=> A > RECEIVED: dog
=> A > SENDING: cat
=> B > RECEIVED: cat
csp.go(function* () {
  const sent1 = 'dog';
  console.log('B > SENDING:', sent1);
  yield csp.put(chA, sent1);

  const rec2 = yield csp.take(chB);
  console.log('B > RECEIVED:', rec2);

  const sent3 = 'WAT!';
  console.log('B > SENDING:', sent3);

});
=> B > SENDING: dog
=> A > RECEIVED: dog
=> A > SENDING: cat
=> B > RECEIVED: cat
=> B > SENDING: WAT!
csp.go(function* () {
  const sent1 = 'dog';
  console.log('B > SENDING:', sent1);
  yield csp.put(chA, sent1);

  const rec2 = yield csp.take(chB);
  console.log('B > RECEIVED:', rec2);

  const sent3 = 'WAT!';
  console.log('B > SENDING:', sent3);
  yield csp.put(chA, sent3);
});
csp.go(function* () {
  const rec1 = yield csp.take(chA);
  console.log('A > RECEIVED:', rec1);

  const sent2 = 'cat';
  console.log('A > SENDING:', sent2);
  yield csp.put(chB, sent2);

  const rec3 = yield csp.take(chA);
  console.log('A > RECEIVED:', rec3);
});
=> B > SENDING: dog
=> A > RECEIVED: dog
=> A > SENDING: cat
=> B > RECEIVED: cat
=> B > SENDING: WAT!
=> A > RECEIVED: WAT!

events

browser events

  • click
  • focus
  • mouse*
  • change
  • etc.

custom events

$(selector).on(event, function(e){
  // use `this` and `e`
});
$someone.on('click', function(){
  $element.trigger('custom');
});

$element.on('custom', function(){
  // logic
});
Backbone.on(event, this.render);
initialize: function() {
  this.listenTo(this.model,
    'change', this.render);
}

RxCpp

Rx.rb

RxPHP

RxJS

RxJava

RxPY

RxKotlin

RxSwift

RxLua

RxClojure

Rx.NET

UniRx

RxScala

RxGroovy

RxGo

RxDart

Array

Stream

[   ,   ,   ,   ,   ]

A

B

C

D

E

,

A

B

C

D

E

,

,

,

entirely in-memory

items pushed over time

completed

[1, 2, 3].map(e => e * 2)
         .map(e => e * 2)
stream$.map(e => e * 2)
  • all available upfront
  • in-memory
  • not necessarily exist
  • sync or async, doesn't matter
  • don't know when they arrive
  • push instead of pull
.subscribe(
  value => console.log(`value: ${value}`),
  ...
);
  • manipulating data
map, filter, find, reduce, sort...

Reactive Streams

  • manipulating data
  • manipulating time
    (when to become available)

Functors/HOF

Arrays

items go down the stream

rx marbles

The Hollywood Principle

const click$ = Rx.Observable.fromEvent(document, 'click');
const click$ = Rx.Observable.fromEvent(document, 'click')
.map(e => ({
  x: e.clientX,
  y: e.clientY
}))
.filter(e => e.x < document.body.clientWidth/2);
click$.subscribe(
  value => console.log(`value: ${value}`),
  e => console.warn(`error: ${e}`),
  () => console.log('completed')
);

observer

+ iterator

+ FP

1. data source

2. processing items

3. observers

Observable streams

are difficult

  • sync or async

  • one or many values

  • still opened or already closed

  • IoC (Holywood principle)

  • observer + iterator + FP

  • observables

  • observables, observers and subscriptions

  • when is the source created: hot and cold observables

  • re-emit: subjects

  • backpressure (lossy, loss-less)

is your app supposed to solve above issues?

demo time

HTTP calls as streams...?

getItems(onNext, onError) {
  this.http
    .get("/items")
    .map(response => response.json())
    .retry(2)
    .subscribe(onNext, onError)
}
@Component({
  selector: 'async-component',
  template: `<code>promise|async</code>
             <code>stream|async</code>`
})
export class AsyncComponent {
  private promise: Promise<any>;
  private stream: Observable<any>;

  constructor(){
    this.promise = new Promise((res, rej) => {
      setTimeout(() => res('item-p'), 1000)
    });

    this.stream = Observable.of('item-s')
      .delay(1000)
  }
}

async pipe 

promise.then(onSuccess, onFailure)
promise.catch(onFailure)
observable$
  .subscribe(onNext, onError, onCompleted)

promise API

observable API

async function process(args){
  let user = await API.getUser(args);
  let address = await API.getAddress(user);
  return { user, address };
}

async await API

functions

sync vs async

promises

async await

coroutines

CSP

generators

callbacks

events

reactive streams

run to completion

event loop

THX!

@tomasz_ducin

tomasz@ducin.it

bit.ly/5-async-js