From procedural to reactive JavaScript with stops

Dr. Gleb Bahmutov PhD

Coding in JS

JavaScript == table

JavaScript < table

We use JS because it is everywhere

  • browser

  • server

  • mobile

  • robots

JavaScript in Tesla?

Scary scary scary

JavaScript in T800?

Humanity quickly wins

JS package growth

Changing how JS developers program will have a huge impact on the software quality in the world

Kensho - instant financial analysis

Harvard Sq, Cambridge MA

WTC New York

Kensho     OSS

Kensho - instant financial analytics

Problem: code complexity is quickly growing

number of engineers

lifetime of the company

We need JavaScript to

  • be as clear as writing an English sentence (or clearer)
  • produce expected result
  • be simple to test
  • assemble complex logic from very simple parts

Use FP to implement Reactive Programming patterns

FP in JS

  • small pure functions
  • compose complex logic from simple steps
  • semi-performant async with promises
  • simple(r) data with immutable structures

reactive programming

  • every user action is stream of events
  • every server action is stream of events
  • streams can be processed using standard operations

2 major parts

Dev team (otherwise smart people)

Everyone has their own path to JS

Everyone is different

  • 100% jQuery
  • 30% FP in JS
  • 25% OO in JS

! junior vs senior

  • 50% jQuery
  • 90% CSS
  • 5% OO in JS
  • 20% jQuery
  • 30% CSS
  • 75% OO in JS
  • 90% FP in JS

In theory

anyone

functional reactive programmer

In practice

anyone

functional reactive programmer

...

...

...

...

...

...

...

  • procedural
  • object-oriented
  • functional
  • abandon return values
  • immutable data
  • lazy evaluation
  • lazy async values
  • promises
  • event emitters
  • event emitters returning promises
  • chained event emitters
  • reactive streams
  • pure reactive streams web applications

Here is my journey through the JS land

Show me the code!

 

Problem: given an array of numbers, multiply each number by a constant and print the result.

var numbers = [3, 1, 7];
var constant = 2;
// 6 2 14

procedural / imperative JS

var numbers = [3, 1, 7];
var constant = 2;
var k = 0;
for(k = 0; k < numbers.length; k += 1) {
  console.log(numbers[k] * constant);
}
// 6 2 14
  • "for" loop is a pain to debug and use

  • mixing data manipulation with printing

  • code is hard to reuse

Refactoring

var numbers = [3, 1, 7];
var constant = 2;

reusable simple functions

complex logic

iteration

function mul(a, b) {
  return a * b;
}
function print(x) {
  console.log(x);
}
function processNumber(n) {
  print(mul(n, constant));
}
for(k = 0; k < numbers.length; k += 1) {
  processNumber(numbers[k]);
}
// 6 2 14

My advice

  1. Write small pure functions

  2. Adapt them to particular call sites

  3. Progress slowly through the code base

Boring code does not surprise you

// using "this" WILL surprise you
numbers.forEach(function (x) {
  this.doSomething(x);
  // ERROR!
});
var self = this;
numbers.forEach(function (x) {
  self.doSomething(x);
});

Boring code does not surprise you

// using "this" in your objects
// WILL surprise someone else
numbers.forEach(console.log);
numbers.forEach(console.log.bind(console));
// works under Node
// in Chrome
// Uncaught TypeError: Illegal invocation

Boring code does not surprise you

// for loops WILL surprise you
for(k = 0; k < numbers.length; k += 1) {
  processNumber(numbers[k]);
}

common problems: off by 1 errors, iterator variable, changing array size

Boring code does not surprise you

// using non-local variables WILL surprise you
constant = 2;
...
numbers.forEach(function process(n) {
  return mul(n, constant);
});

Pure functions are really boring

function mul(a, b) {
  return a * b;
}
function print(n) {
  console.log(n);
}
  • output depends ONLY on the inputs
    • same result every time
  • no outside state
  • pure against set of unit tests

not really

pure but

close enough for now

Boring functions make new boring functions using fn.bind

var mulBy2 = mul.bind(null, 2);
mulBy2(5); // 10
function mul(a, b) {
  return a * b;
}
var mul2by3 = mul.bind(null, 2, 3);
mul2by3(); // 6

Partial application

function mul(a, b) {
  return a * b;
}
var by2 = mul.bind(null, 2);
// same as
var _ = require('lodash');
var by2 = _.partial(mul, 2);

Boring code is nice, but I want encapsulated logic, data access, inheritance!

function NumberMultiplier() {}

Object-Oriented JavaScript

NumberMultiplier.prototype.setNumbers = function (numbers) {
    this.numbers = numbers;
    return this;
};
NumberMultiplier.prototype.multiply = function (constant) {
    for (var k = 0; k < this.numbers.length; k += 1) {
      this.numbers[k] = constant * this.numbers[k];
    }
    return this;
};
NumberMultiplier.prototype.print = function () {
    console.log(this.numbers);
};

Object-Oriented JavaScript

// using NEW keyword
new NumberMultiplier()
  .setNumbers(numbers)
  .multiply(constant)
  .print();
// [ 6, 2, 14 ]

Object-Oriented JavaScript: 3 parts

function NumberMultiplier() {}

NumberMultiplier.prototype.setNumbers = function() {}
NumberMultiplier.prototype.multiply = function() {}
NumberMultiplier.prototype.print = function() {}

new NumberMultiplier();

constructor function

prototype

"new" keyword

// NumberMultiplier.prototype object
{
  setNumbers: function () { ... },
  multiply: function () { ... },
  print: function () { ... }
}

OO JavaScript is prototypical

// instance object
{
  numbers: [3, 1, 7]
}

prototype

var NumberMultiplier = {
  setNumbers: function (numbers) {
    this.numbers = numbers; return this;
  },
  multiply: function (constant) {
    for (var k = 0; k < this.numbers.length; k += 1) {
      this.numbers[k] = constant * this.numbers[k];
    } return this;
  },
  print: function () { console.log(this.numbers); }
};

Prototype is a plain object!

var numberMultiplier = Object.create(NumberMultiplier);
numberMultiplier
  .setNumbers(numbers)
  .multiply(constant)
  .print();
// [ 6, 2, 14 ]

plain object

class NumberMultiplier {
  setNumbers(numbers) {
    this.numbers = numbers;
    return this;
  }
  multiply(constant) {
    for (var k = 0; k < this.numbers.length; k += 1) {
      this.numbers[k] = constant * this.numbers[k];
    }
    return this;
  }
  print() {
    console.log(this.numbers);
    return this;
  }
}
new NumberMultiplier()

ES6 classes: just don't

Observation: a lot of our code

manipulates lists items

JavaScript Arrays

typeof [1, 2] // "object"
Array.isArray([1, 2]) // true
Array.prototype
/*
  map
  forEach
  filter
  reduce
  ...
*/

Arrays: object-oriented + functional

var numbers = [3, 1, 7];
var constant = 2;
function mul(a, b) {
  return a * b;
}
function print(n) {
  console.log(n);
}
var mulBy = mul.bind(null, constant);

clear "multiply then print" semantics

functional bits

Object-oriented methods

numbers
    .map(mulBy)
    .forEach(print);
// 6 2 14
// OO (built-in)
Array.prototype.map(cb)

// functional (user space)
Library.map(array, cb)

// plus
Library.groupBy(array, cb);
Library.dropWhile(array, cb);
...

2 of the top 3 most dependent upon NPM projects are FP utility libraries: underscorelodash

var _ = require('lodash')
var byConstant = _.partial(mul, constant)

Work with arrays using Lodash

var multiplied = _.map(numbers, byConstant)
_.forEach(multiplied, print)
// 6 2 14
_({ John: 3, Jim: 10 })
  .map(byConstant)
  .forEach(print);
// 6 20

Treat objects same as collections using lodash

Argument order

var mulBy = ...

// OO (built-in)
Array.prototype.map(mulBy)

// lodash / underscore
_.map(array, mulBy)
// Ramda
R.map(mulBy, array)

Callback-first order

will make composing functions easier

// Ramda
R.map(mulBy, array)
var R = require('ramda');
var byConstant = R.partial(R.mul, constant);

1. compose algorithm

2. apply to data

3. profit!!!

// R.forEach = function (callback, array)
var printEach = R.partial(R.forEach, print);
// R.map: function (callback, array)
var mapByConstant = R.partial(R.map, byConstant);
printEach(mapByConstant(numbers));
// 6 2 14
printEach(mapByConstant(numbers));
// f ( g ( x ) )

Composition of functions

var algorithm = R.compose(printEach, mapByConstant);
algorithm(numbers);
// 6 2 14
// fn( data )
algorithm(numbers);

"fn(data)" code is simple to reason about and test

// fn( data )
// R.mul = function (a, b) { ... }
var byConstant = R.partial(R.mul, constant);

Partial application by the client

// in Ramda this is same as simply
var byConstant = R.mul(constant);
// for your functions
var mul = R.curry(function (a, b) {
  return a * b; 
});
mul(5)(4); // 20

Curry: partial application built-in

var byConstant = R.mul(constant);
R.map(R.mul(constant)) (... data ...)

Ramda: library with all functions curried

var R = require('ramda');
R.pipe(
  R.map(R.multiply(constant)),
  R.forEach(print)
)(numbers);
// 6 2 14

Curry: program with very little code

Requires careful function signature design

i.e. put information most likely to be known early as first argument

OO

"Smart" objects

Pass data around

FP

"Dumb" objects

"Smart" pipeline

Pass code (functions) around

new NumberMultiplier()
    .add(3, 1, 7)
    .multiply(2)
    .print();
_.forEach(
  _.map(numbers, byConstant),
  print
);

Object-oriented VS functional

It is better to have 100 functions operate on one data structure than to have 10 functions operate on 10 data structures."

                                  - Alan J. Perlis

new NumberMultiplier()
    .add(3, 1, 7)
    .multiply(2)
    .print();
_.forEach(
  _.map(numbers, byConstant),
  print
);

What else can JavaScript do for us?

Source of complexity: mutable data

var numbers = [...];
numbers.forEach(processNumber);
// is the list "numbers" unchanged?
function processNumber(x, k, array) {
    array[1] = 10000;
    print(x);
}

Mutable data makes code hard to understand and modify

Immutable data structures

'use strict';
var immutable = require('seamless-immutable');
var byConstant = _.partial(mul, constant);
immutable(numbers)
  .map(byConstant)
  .forEach(function (x, k, array) {
    array[1] = 10000; // throws an Error
    print(x);
  });

Efficient immutable JS libraries: seamless-immutablemori

Immutable data with Redux pattern

function mul(a, b) { return a * b; }

function multiply(state, constant) {
  var byConstant = mul.bind(null, constant);
  return state.updateIn(['numbers'], function (ns) {
    return ns.map(byConstant);
  });
}
var immutable = require('immutable');
var initialState = immutable.Map({ 
  numbers: immutable.List.of([3, 1, 7])
});
var newState = multiply(initialState, constant);
// newState !== initialState

Arrays: great for synchronous actions

Arrays: NOT SO great for asynchronous

or infinite data!

// Does NOT work
numbers.map(function (x, done) {
  mul(x, function cb(result) {
    done(result);
  })
})

How to handle async callbacks?

numbers
  .map(asyncFn)
  .forEach(asyncFn);

Promises

var Q = require('q');
Q(...)
    .then(function onSuccess(result) { ... },
          function onFailure(error) { ... })
    .then(...);

A single async action with a single result or error

var Q = require('q');
function foo() {
  var defer = Q.defer();
  ... 
    defer.resolve(returnValue);
  ...
  return defer.promise;
}
foo()
    .then(bar)
    .then(baz);
// functions foo, bar, baz return a promise

Each step in the promise chain can be asynchronous and return a promise!

var sleepSecond = _.partial(Q.delay, 1000);

var pauseMulAndPrint = function (n) {
  return function () {
    return sleepSecond()
        .then(_.partial(byConstant, n))
        .then(print);
  };
};

Sleep, mul then print using promises

numbers.map(pauseMulAndPrint)
  .reduce(Q.when, Q())
  .done();
// ... 6 ... 2 ... 14

Promises handle single event very well

Extra complexity trying to handle sequence of events

var sleepSecond = _.partial(Q.delay, 1000);

var pauseMulAndPrint = function (n) {
  return function () {
    return sleepSecond()
        .then(_.partial(byConstant, n))
        .then(print);
  };
};

numbers.map(pauseMulAndPrint)
  .reduce(Q.when, Q())
  .done();
// ... 6 ... 2 ... 14

Constructing a promise for each number :(

var sleepSecond = _.partial(Q.delay, 1000);

var pauseMulAndPrint = function (n) {
  return function () {
    return sleepSecond()
        .then(_.partial(byConstantAsync, n))
        .then(print);
  };
};

Promise is like a single cartridge

Improve upon promises: need to chain async actions without boilerplate per item

"shoot the same bullet more than once"

Event Emitters

  • handle repeated events

  • decouple the event source from an action

Widely used in browser code

$(el).on('click', action)

window.on('resize', cb)

socket.on('message', handle)

We can extend the same approach to any event generator (like number sequence)

// action
var events = require('events');
var numberEmitter = new events.EventEmitter();
numberEmitter.on('number', 
  _.compose(print, byConstant)
);

Single code pipeline

var k = 0;
var ref = setInterval(function () {
  numberEmitter.emit('number', numbers[k++]);
  if (k >= numbers.length) {
    clearInterval(ref);
  }
}, 1000);
// prints 6, 2 14 with 1 second intervals

Goal: process number with 1 second pauses

Our emitter has `on`

  1. What is the semantic meaning of `on`?
  2. How do we connect multiple steps?
numberEmitter
  .on('number', action);

Remember Array ES5 methods?

[3, 1, 7]
  .map(mulBy2)
  .forEach(print);

I want this!

source of number events
  .map(an async callback)
  .buffer(n)
  .forEach(another async callback);

pipeline!

source of number events

source

.map

fn

.buffer

3

.forEach

fn

  .forEach(an async callback)
  .map(an async callback)
  .buffer(3)

pipeline working

source of number events

source

.map

fn

.buffer

3

.forEach

fn

  .forEach(an async callback)
  .map(an async callback)
  .buffer(3)

3

6

6,

pipeline working

source of number events

.map

fn

.buffer

3

.forEach

fn

  .forEach(an async callback)
  .map(an async callback)
  .buffer(3)

1

2

2,

source

6,

pipeline working

source of number events

.map

fn

.buffer

3

.forEach

fn

  .forEach(an async callback)
  .map(an async callback)
  .buffer(3)

7

14

14

source

2,

6,

pipeline working

source of number events

.map

fn

.buffer

3

.forEach

fn

  .forEach(an async callback)
  .map(an async callback)
  .buffer(3)

6, 2, 14

source

Building pipeline with emitters

var stepEmitter = {









};
stepEmitter(numbers)
  .map(an async callback)
  map: function (cb) {
    var emitter = new events.EventEmitter();




    return _.extend(emitter, stepEmitter);
  }
var stepEmitter = {









};
stepEmitter(numbers)
  .map(an async callback)
  map: function (cb) {
    var emitter = new events.EventEmitter();
    this.on('step', function (value) {
      Q.when(cb(value))
        .then(output => emitter.emit('step', output))
    });
    return _.extend(emitter, stepEmitter);
  }
var stepEmitter = {
  // same map and forEach methods as above

  // returns a new step emitter that accumulates N items
  // emits the entire array with N items as single argument
  buffer: function (n) {
    var received = [];
    var emitter = new events.EventEmitter();
    this.on('step', function (value) {
      received.push(value);
      if (received.length === n) {
        emitter.emit('step', received);
        received = [];
      }
    });
    return _.extend(emitter, stepEmitter);
  }
};
source(numbers)
  .map(byConstant)
  .buffer(3)
  .forEach(print);
// sleeps 3 seconds then prints [6, 2, 14]

What about buffering events?

source(numbers)
  .map(byConstant) // data transform
  .buffer(3)       // sequence control
  .forEach(print); // data transform

We implemented 2 types of operations on a sequence of events

Need extra methods to control the sequence (stream) of events

.map, .filter
// vs
.buffer

Reactive programming

  • Everything is a source of asynchronous events.
  • Your code is a pipeline for processing these events

Reactive JS libraries

  • collection processing methods (map, filter, ...)
  • stream control methods (buffer, zip, flatMapLatest, ...)

Use a reactive FP library

var Rx = require('rx');

var timeEvents = Rx.Observable
  .interval(1000)
  .timeInterval(); // stream 1
var numberEvents = Rx.Observable
  .fromArray(numbers); // stream 2
Rx.Observable.zip(timeEvents, numberEvents, 
                  function pickValue(t, n) { return n; })
  .map(byConstant)
  .subscribe(print)

// prints 6 2 14 with 1 second intervals

User click events

var Rx = require('rx');

var clickEvents = Rx.Observable
  .fromEvent(document.querySelector('#btn'), 'click')

var numberEvents = Rx.Observable
  .fromArray(numbers);

function pickSecond(c, n) { return n; }

Rx.Observable.zip(clickEvents, numberEvents, 
                  pickSecond)
  .map(byConstant)
  .subscribe(print);
// prints a number on each button click

Marble diagram for 

zip(s1, s2, fn)
function searchWikipedia (term) {
    return $.ajax({
        url: 'http://en.wikipedia.org/w/api.php',
        dataType: 'jsonp',
        data: {
            action: 'opensearch',
            format: 'json',
            search: term
        }
    }).promise();
}

Autocomplete

Hard problem: need to throttle, handle out of order returned results, etc.

var keyups = Rx.Observable.fromEvent($('#input'), 'keyup')
    .map(function (e) { return e.target.value; })
    .filter(function (text) { return text.length > 2; });

Autocomplete with RxJs

keyups.throttle(500)
    .distinctUntilChanged()
    .flatMapLatest(searchWikipedia)
    .subscribe(function (data) {
        // display results
    });

Pure functional Reactive Programming

function add(a, b) {
  return a + b
}

Remember pure functions?

const F = compose(f, g, h)
F(x)

pure

pure

Can the entire web application be pure?

Getting dirty

function main() {
  window.title = 'hi'
}

Pure function loophole

function main() {
  const title = 'hi'
  return function foo() {
    window.title = title
  }
}

// pure

// dirty

const app = main()
app()

// pure

// dirty

A little better

function main(win) {
  const title = 'hi'
  return function foo() {
    win.title = title
  }
}

// pure

// dirty

const app = main(window)
app()

// pure

// dirty

Cycle.js = pure FP + RxJs

function main({DOM}) {
  const keyups$ = DOM.select(...)
  return {
    DOM: keyups$.map(...vdom...)
  }
}

all inputs are streams

all outputs are streams

Cycle.js = pure FP + RxJs

function main({DOM}) {
  const keyups$ = DOM.select(...)
  return {
    DOM: keyups$.map(...vdom...)
  }
}

// pure

Cycle.js = pure FP + RxJs

Cycle.run(main, {
  DOM: makeDomDriver... 
})

// dirty

Summary

From procedural to RFP

Write small pure functions

Summary

From procedural to RFP

​Handle single async events using promises

Summary

From procedural to RFP

​Handle multiple async event streams using Rx

Future is bright

All examples shown use today's ES5 standard

ES6 (EcmaScript2015) will make some pieces simpler

  • arrow notation
  • block-scoping and constants
  • spread operator
  • native Promises
  • binding context in callbacks

Thank you

#OSCON @bahmutov

Kensho     OSS

From procedural to reactive JavaScript with stops - O'Reilly OSCON

By Gleb Bahmutov

From procedural to reactive JavaScript with stops - O'Reilly OSCON

How do we convince / train developers used to imperative programming style to switch to functional or even reactive programming? I show the same example implemented in multiple styles to highlight the advantages one would get by going from imperative to object-oriented, functional, lazy evaluation, promise-based, event emitters, and finally reactive approach. The short examples are in JavaScript - the most widely used functional language today.

  • 3,041

More from Gleb Bahmutov