Journey from procedural to reactive JavaScript with stops

Dr. Gleb Bahmutov PhD

Kensho Boston / NYC

Boston Code Camp 24 #boscc

Coding in JS

source: Improved Minesweeper

JavaScript == table

source: Don't Make Javascript Equality Look Worse Than It Is 

JavaScript < table

source: Don't Make Javascript Equality Look Worse Than It Is 

We use JS because it is everywhere

  • browser

  • server

  • mobile

  • robots

cyclon.js

source: careers.stackoverflow.com

JS package growth

source: modulecounts.com

Changing how JS developers program will have a huge impact on the software quality in the world

Kensho - instant financial analysis

Harvard Sq, Cambridge MA

WTC New York

Kensho - instant financial analytics

Problem: code complexity is quickly growing

number of engineers

lifetime of the company

We need JavaScript to

  • be as clear as writing an English sentence (or clearer)
  • produce expected result
  • be simple to test
  • assemble complex logic from very simple parts

Use FP to implement Reactive Programming patterns

reactivex.io/tutorials.html

js books

FP in JS

  • small pure functions
  • compose complex logic from simple steps
  • semi-performant async with promises
  • simple(r) data with immutable structures

reactive programming

  • every user action is stream of events
  • every server action is stream of events
  • streams can be processed using standard operations

2 major parts

Dev team (otherwise smart people)

Everyone has their own path to JS

Everyone is different

  • 100% jQuery
  • 30% FP in JS
  • 25% OO in JS

! junior vs senior

  • 50% jQuery
  • 90% CSS
  • 5% OO in JS
  • 20% jQuery
  • 30% CSS
  • 75% OO in JS
  • 90% FP in JS

In theory

anyone

functional reactive programmer

In practice

anyone

functional reactive programmer

...

...

...

...

...

...

...

  • procedural
  • object-oriented
  • functional
  • abandon return values
  • immutable data
  • lazy evaluation
  • lazy async values
  • promises
  • event emitters
  • event emitters returning promises
  • chained event emitters
  • reactive streams

Here is my journey through the JS land

Show me the code!

 

Problem: given an array of numbers, multiply each number by a constant and print the result.

var numbers = [3, 1, 7];
var constant = 2;
// 6 2 14

more than 20 implementations at 

github.com/bahmutov/javascript-journey

github.com/bahmutov/javascript-journey

procedural / imperative JS

var numbers = [3, 1, 7];
var constant = 2;
var k = 0;
for(k = 0; k < numbers.length; k += 1) {
  console.log(numbers[k] * constant);
}
// 6 2 14

  • "for" loop is a pain to debug and use

  • mixing data manipulation with printing​

    • code is hard to reuse

Programming without for loops

Start splitting code into procedures

var numbers = [3, 1, 7];
var constant = 2;
function mul(a, b) {
  return a * b;
}
function print(x) {
  console.log(x);
}
function processNumber(n) {
  print(mul(n, constant));
}
for(k = 0; k < numbers.length; k += 1) {
  processNumber(numbers[k]);
}
// 6 2 14

reusable simple functions

complex logic

iteration

Our goal

make the code boring

Boring code does not surprise you

// using "this" WILL surprise you
numbers.forEach(function (x) {
  this.doSomething(x);
  // ERROR!
});
var self = this;
numbers.forEach(function (x) {
  self.doSomething(x);
});

Boring code does not surprise you

// using "this" in your objects
// WILL surprise someone else
numbers.forEach(console.log);
numbers.forEach(console.log.bind(console));
// works under Node
// in Chrome
// Uncaught TypeError: Illegal invocation

Boring code does not surprise you

// for loops WILL surprise you
for(k = 0; k < numbers.length; k += 1) {
  processNumber(numbers[k]);
}

common problems: off by 1 errors, iterator variable, changing array size

Boring code does not surprise you

// using non-local variables WILL surprise you
constant = 2;
...
numbers.forEach(function process(n) {
  return mul(n, constant);
});

Pure functions are really boring

function mul(a, b) {
  return a * b;
}
function print(n) {
  console.log(n);
}

Boring functions make new boring functions using fn.bind

var mulBy2 = mul.bind(null, 2);
mulBy2(5); // 10

Binding vs partial application

function mul(a, b) {
  return a * b;
}

var mul2by3 = mul.bind(null, 2, 3);
mul2by3(); // 6
var mul2By10 = mulBy2.bind(null, 10);
mul2By10('extra', 'args are', 'ignored'); // 20

Partial application

function mul(a, b) {
  return a * b;
}
var by2 = mul.bind(null, 2);
// same as
var _ = require('lodash');
var by2 = _.partial(mul, 2);

_.partial, _.partialRight, spots, obind, heroin

Boring code is nice, but I want encapsulated logic, data access, inheritance!

function NumberMultiplier() {}

NumberMultiplier.prototype.setNumbers = function (numbers) {
    this.numbers = numbers;
    return this;
};
NumberMultiplier.prototype.multiply = function (constant) {
    for (var k = 0; k < this.numbers.length; k += 1) {
      this.numbers[k] = constant * this.numbers[k];
    }
    return this;
};
NumberMultiplier.prototype.print = function () {
    console.log(this.numbers);
};

Object-Oriented JavaScript

Object-Oriented JavaScript

// using NEW keyword
new NumberMultiplier()
  .setNumbers(numbers)
  .multiply(constant)
  .print();
// [ 6, 2, 14 ]

Object-Oriented JavaScript: 3 parts

function NumberMultiplier() {}

NumberMultiplier.prototype.setNumbers = function() {}
NumberMultiplier.prototype.multiply = function() {}
NumberMultiplier.prototype.print = function() {}

new NumberMultiplier();

constructor function

prototype

"new" keyword

// NumberMultiplier.prototype object
{
  setNumbers: function () { ... },
  multiply: function () { ... },
  print: function () { ... }
}
  /\
  || [[ prototype ]]
  ||
// instance object
{
  numbers: [3, 1, 7]
}

OO JavaScript is prototypical

Common Misconceptions About Inheritance in JavaScript

var NumberMultiplier = {
  setNumbers: function (numbers) {
    this.numbers = numbers; return this;
  },
  multiply: function (constant) {
    for (var k = 0; k < this.numbers.length; k += 1) {
      this.numbers[k] = constant * this.numbers[k];
    } return this;
  },
  print: function () { console.log(this.numbers); }
};

Prototype is a plain object!

javascript-journey source

var numberMultiplier = Object.create(NumberMultiplier);
numberMultiplier
  .setNumbers(numbers)
  .multiply(constant)
  .print();
// [ 6, 2, 14 ]

plain object

class NumberMultiplier {
  setNumbers(numbers) {
    this.numbers = numbers;
    return this;
  }
  multiply(constant) {
    for (var k = 0; k < this.numbers.length; k += 1) {
      this.numbers[k] = constant * this.numbers[k];
    }
    return this;
  }
  print() {
    console.log(this.numbers);
    return this;
  }
}
new NumberMultiplier()

ES6 classes: just don't

How to Fix the ES6 `class` keyword

JavaScript Arrays

typeof [1, 2] // "object"
Array.isArray([1, 2]) // true
Array.prototype
/*
  map
  forEach
  filter
  reduce
  ...
*/

Arrays: object-oriented + functional

var numbers = [3, 1, 7];
var constant = 2;
function mul(a, b) {
  return a * b;
}
function print(n) {
  console.log(n);
}
var mulBy = mul.bind(null, constant);
numbers
    .map(mulBy)
    .forEach(print);
// 6 2 14
  • clear "multiply then print" semantics

functional bits

Object-oriented methods

My advice

  1. Write small pure functions

  2. Adapt them to particular call sites

  3. Progress slowly through the code base

flexible selective application

Observation: a lot of our code

manipulates collections of data items

// OO (built-in)
Array.prototype.map(cb)


// functional (user space)
Library.map(array, cb)

// plus
Library.groupBy(array, cb);
Library.dropWhile(array, cb);
...

2 of the top 3 most dependent upon NPM projects are FP utility libraries: underscore, lodash 

var _ = require('lodash');
var byConstant = _.partial(mul, constant);
// _.forEach(array, cb);
// _.map(array, cb);

Work with arrays using Lodash

_.forEach(
  _.map(numbers, byConstant),
  print
);
// 6 2 14
_({ John: 3, Jim: 10 })
  .map(byConstant)
  .forEach(print);
// 6 20

Treat objects same as collections using lodash

object iterators

Argument order

var mulBy = ...

// OO (built-in)
Array.prototype.map(mulBy)

// lodash / underscore
_.map(array, mulBy)

Put callback first for elegance

// Ramda
R.map(mulBy, array)
var R = require('ramda');
var byConstant = R.partial(R.mul, constant);

1. compose algorithm

2. apply to data

3. profit!!!

// R.forEach = function (callback, array)
var printEach = R.partial(R.forEach, print);
// R.map: function (callback, array)
var mapByConstant = R.partial(R.map, byConstant);

composition

printEach(mapByConstant(numbers));
// 6 2 14
var R = require('ramda');
var byConstant = R.partial(R.mul, constant);

1. compose algorithm

2. apply to data

3. profit!!!

// R.forEach = function (callback, array)
var printEach = R.partial(R.forEach, print);
var algorithm = R.compose(printEach, mapByConstant);

algorithm(numbers);
// 6 2 14
// R.map: function (callback, array)
var mapByConstant = R.partial(R.map, byConstant);
// fn( data )
var R = require('ramda');
var byConstant = R.partial(R.mul, constant);
// R.forEach = function (callback, array)
var printEach = R.partial(R.forEach, print);
var algorithm = R.compose(printEach, mapByConstant);

algorithm(numbers);
// 6 2 14
// R.map: function (callback, array)
var mapByConstant = R.partial(R.map, byConstant);

"fn(data)" code is simple to reason about and test

// fn( data )
// R.mul = function (a, b) { ... }
var byConstant = R.partial(R.mul, constant);

Partial application vs curry

// in Ramda this is same as simply
var byConstant = R.mul(constant);

// for your functions
var mul = R.curry(function (a, b) {
  return a * b; 
});
mul(5)(4); // 20

var R = require('ramda');
R.pipe(
  R.map(R.multiply(constant)),
  R.forEach(print)
)(numbers);
// 6 2 14

Curry: program with very little code

Requires careful function signature design

i.e. put information most likely to be known early as first argument

User friendly API

OO

"Smart" objects

Pass data around

FP

"Dumb" objects

"Smart" pipeline

Pass code (functions) around

new NumberMultiplier()
    .add(3, 1, 7)
    .multiply(2)
    .print();
_.forEach(
  _.map(numbers, byConstant),
  print
);

Object-oriented VS functional

It is better to have 100 functions operate on one data structure than to have 10 functions operate on 10 data structures."

                                  - Alan J. Perlis

new NumberMultiplier()
    .add(3, 1, 7)
    .multiply(2)
    .print();
_.forEach(
  _.map(numbers, byConstant),
  print
);

What else can JavaScript do for us?

Source of complexity: mutable data

var numbers = [...];
numbers.forEach(processNumber);
// is the list "numbers" unchanged?
function processNumber(x, k, array) {
    array[1] = 10000;
    print(x);
}

Mutable data makes code hard to understand and modify

Immutable data structures

'use strict';
var immutable = require('seamless-immutable');
var byConstant = _.partial(mul, constant);
immutable(numbers)
  .map(byConstant)
  .forEach(function (x, k, array) {
    array[1] = 10000; // throws an Error
    print(x);
  });

Efficient immutable JS libraries: seamless-immutable, mori

Immutable data with Redux pattern

function mul(a, b) { return a * b; }

function multiply(state, constant) {
  var byConstant = mul.bind(null, constant);
  return state.updateIn(['numbers'], function (ns) {
    return ns.map(byConstant);
  });
}

see the Redux example in javascript-journey

var immutable = require('immutable');
var initialState = immutable.Map({ 
  numbers: immutable.List.of([3, 1, 7])
});
var newState = multiply(initialState, constant);
// newState !== initialState

Arrays: great for synchronous actions

Efficient computation with transducers

Do as little as needed with lazy evaluation

Arrays: NOT SO great for asynchronous

or infinite data!

// Does NOT work
numbers.map(function (x, done) {
  mul(x, function cb(result) {
    done(result);
  })
}).forEach(function (value) {
  print(value, function cb() {
    done():
  })
});

How to handle async callbacks?

numbers
  .map(asyncFn)
  .forEach(asyncFn);

Promises

var Q = require('q');
Q(...)
    .then(function onSuccess(result) { ... },
          function onFailure(error) { ... })
    .then(...);

A single async action with a single result or error

var Q = require('q');
function foo() {
  var defer = Q.defer();
  ... 
    defer.resolve(returnValue);
  ...
  return defer.promise;
}
foo()
    .then(bar)
    .then(baz);
// functions foo, bar, baz return a promise

Each step in the promise chain can be asynchronous and return a promise!

chaining promises

var sleepSecond = _.partial(Q.delay, 1000);

var pauseMulAndPrint = function (n) {
  return function () {
    return sleepSecond()
        .then(_.partial(byConstant, n))
        .then(print);
  };
};

Sleep, mul then print using promises

numbers.map(pauseMulAndPrint)
  .reduce(Q.when, Q())
  .done();
// ... 6 ... 2 ... 14

chaining promises

Promises handle single event very well

Extra complexity trying to handle sequence of events

var sleepSecond = _.partial(Q.delay, 1000);

var pauseMulAndPrint = function (n) {
  return function () {
    return sleepSecond()
        .then(_.partial(byConstant, n))
        .then(print);
  };
};

numbers.map(pauseMulAndPrint)
  .reduce(Q.when, Q())
  .done();
// ... 6 ... 2 ... 14

Constructing a promise for each number :(

var sleepSecond = _.partial(Q.delay, 1000);

var pauseMulAndPrint = function (n) {
  return function () {
    return sleepSecond()
        .then(_.partial(byConstantAsync, n))
        .then(print);
  };
};

Promise is like a single cartridge

Improve upon promises: need to chain async actions without boilerplate per item

"shoot the same bullet more than once"

Event Emitters

  • handle repeated events

  • decouple the event source from an action

Widely used in browser code

$(el).on('click', action)

window.on('resize', cb)

socket.on('message', handle)

We can extend the same approach to any event generator (like number sequence)

// action
var events = require('events');
var numberEmitter = new events.EventEmitter();
numberEmitter.on('number', 
  _.compose(print, byConstant)
);

Single code pipeline

var k = 0;
var ref = setInterval(function () {
  numberEmitter.emit('number', numbers[k++]);
  if (k >= numbers.length) {
    clearInterval(ref);
  }
}, 1000);
// prints 6, 2 14 with 1 second intervals

Goal: process number with 1 second pauses

Our emitter has `on`

  1. What is the semantic meaning of `on`?
  2. How do we connect multiple steps?
numberEmitter
  .on('number', action);

Remember Array ES5 methods?

[3, 1, 7]
  .map(mulBy2)
  .forEach(print);

I want this!

source of number events
  .map(an async callback)
  .buffer(n, async callback)
  .forEach(another async callback);

Chained emitters

var stepEmitter = {

  map: function (cb) {
    var emitter = new events.EventEmitter();




    return _.extend(emitter, stepEmitter);
  },

  forEach: function (cb) {
    var emitter = new events.EventEmitter();




    return _.extend(emitter, stepEmitter);
  }
};

Chained emitters

var stepEmitter = {

  map: function (cb) {
    var emitter = new events.EventEmitter();
    this.on('step', function (value) {
      var mappedValue = cb(value);
      emitter.emit('step', mappedValue);
    });
    return _.extend(emitter, stepEmitter);
  },

  forEach: function (cb) {
    var emitter = new events.EventEmitter();
    this.on('step', function (value) {
      cb(value);
      emitter.emit('step', value);
    });
    return _.extend(emitter, stepEmitter);
  }
};

Chained emitters

var stepEmitter = {

  map: function (cb) {
    var emitter = new events.EventEmitter();
    this.on('step', function (value) {
      var mappedValue = cb(value);
      emitter.emit('step', mappedValue);
    });
    return _.extend(emitter, stepEmitter);
  },

  forEach: function (cb) {
    var emitter = new events.EventEmitter();
    this.on('step', function (value) {
      cb(value);
      emitter.emit('step', value);
    });
    return _.extend(emitter, stepEmitter);
  }
};

.map

emits cb(x)

.forEach

emits x

source of number events
  .map(syncFn) 
  .forEach(syncFn);

Chained emitters

So far: steps are async, but each callback function is synchronous!

Async callback support via promises

var Q = require('q');
var stepEmitter = {
  map: function (cb) {
    var emitter = new events.EventEmitter();
    this.on('step', function (value) {
      Q.when(cb(value)).then(function (mappedValue) {
        emitter.emit('step', mappedValue);
      });
    });
    return _.extend(emitter, stepEmitter);
  },

  forEach: function (cb) {
    var emitter = new events.EventEmitter();
    this.on('step', function (value) {
      Q.when(cb(value)).then(function () {
        emitter.emit('step', value);
      });
    });
    return _.extend(emitter, stepEmitter);
  }
};
source of number events
  .map(asyncFn) 
  .forEach(asyncFn);

Chained emitters + promises

async for the win!

var stepEmitter = {
  // same map and forEach methods as above

  // returns a new step emitter that accumulates N items
  // emits the entire array with N items as single argument
  buffer: function (n) {
    var received = [];
    var emitter = new events.EventEmitter();
    this.on('step', function (value) {
      received.push(value);
      if (received.length === n) {
        emitter.emit('step', received);
        received = [];
      }
    });
    return _.extend(emitter, stepEmitter);
  }
};
source(numbers)
  .map(byConstant)
  .buffer(3)
  .forEach(print);
// sleeps 3 seconds then prints [6, 2, 14]

What about buffering events?

source(numbers)
  .map(byConstant) // data transform
  .buffer(3)       // sequence control
  .forEach(print); // data transform

We implemented 2 types of operations on a sequence of events

Need extra methods to control the sequence (stream) of events

.map, .filter
// vs
.buffer

Reactive programming

  • Everything is a source of asynchronous events.
  • Your code is a pipeline reacting to one or more input streams and outputting another stream of events

Reactive JS libraries

  • collection processing methods (map, filter, ...)
  • stream control methods (buffer, zip, flatMapLatest, ...)

RxJs, BaconJS, cycle

Use a reactive FP library

var Rx = require('rx');

var timeEvents = Rx.Observable
  .interval(1000)
  .timeInterval(); // stream 1
var numberEvents = Rx.Observable
  .fromArray(numbers); // stream 2
Rx.Observable.zip(timeEvents, numberEvents, 
                  function pickValue(t, n) { return n; })
  .map(byConstant)
  .subscribe(print); // stream 3

// prints 6 2 14 with 1 second intervals

javascript-journey source

User click events

var Rx = require('rx');

var clickEvents = Rx.Observable
  .fromEvent(document.querySelector('#btn'), 'click')

var numberEvents = Rx.Observable
  .fromArray(numbers);

function pickSecond(c, n) { return n; }

Rx.Observable.zip(clickEvents, numberEvents, 
                  pickSecond)
  .map(byConstant)
  .subscribe(print);
// prints a number on each button click

Marble diagram for 

Rx.zip

zip(s1, s2, fn)
function searchWikipedia (term) {
    return $.ajax({
        url: 'http://en.wikipedia.org/w/api.php',
        dataType: 'jsonp',
        data: {
            action: 'opensearch',
            format: 'json',
            search: term
        }
    }).promise();
}

Autocomplete

Hard problem: need to throttle, handle out of order returned results, etc.

function searchWikipedia (term) {
    return $.ajax({
        url: 'http://en.wikipedia.org/w/api.php',
        dataType: 'jsonp',
        data: {
            action: 'opensearch',
            format: 'json',
            search: term
        }
    }).promise();
}
var keyups = Rx.Observable.fromEvent($('#input'), 'keyup')
    .map(function (e) { return e.target.value; })
    .filter(function (text) { return text.length > 2; });

Autocomplete

keyups.throttle(500)
    .distinctUntilChanged()
    .flatMapLatest(searchWikipedia)
    .subscribe(function (data) {
        // display results
    });

Rx is the machine gun,

events are the ammo

Summary

Write small pure functions

From procedural to RFP

Summary

​Handle single async events using promises

From procedural to RFP

Summary

​Handle multiple async event streams using Rx

From procedural to RFP

Say "Goodbye" to old friends

Tell return values Bye-bye

// imperative code
var sum = add(2, 3);
// can use sum right away!

var multiplied = [1, 2, 3].map(mul);
// can use multiplied array right away!
var _ = require('lodash');
var byConstant = _.partial(mul, constant);
_(numbers)
  .map(byConstant)
  .forEach(print);
// we have not returned any values!

Abandoning return values

Tell middle functions Bye-bye

var byConstant = _.partial(mul, constant);
_(numbers)
  .map(function (x) {
    return byConstant(x);
  })
  .forEach(function (x) {
    print(x);
  });
_(numbers)
  .map(byConstant)
  .forEach(print);
// no middle functions

Point-free is not pointless

Tell deep equality checks Bye-bye*

var result = process(numbers);
if (deepEqual(result, numbers)) {
  render();
}
var result = process(immutableNumbers);
if (result === immutableNumbers) {
  render();
}

Avoid side effects

* with immutable data

Q: Which step causes the most difficulties?

A: For me it was abandoning the return values, others had problem with promises

Abandoning return values

Q: How to nudge developers along the journey?

A: Having everyone interested train together

Overcoming functional

programming objections

Kensho progress report

object-oriented        **             2 /10
functional             *********      9 /10

immutable data         ******         6 /10
lazy evaluation        *              1 /10

promises               *********      9 /10
event emitters         *****          5 /10
reactive streams       *********      9 /10

Future is bright

All examples shown use today's ES5 standard

ES6 (EcmaScript2015) will make some pieces simpler

  • arrow notation
  • block-scoping and constants
  • spread operator
  • native Promises
  • binding context in callbacks

Dr. Gleb Bahmutov PhD   Kensho

Boston Code Camp 24 #boscc

Journey from imperative to reactive JavaScript with stops - Boston Code Camp 24

By Gleb Bahmutov

Journey from imperative to reactive JavaScript with stops - Boston Code Camp 24

How do we convince / train developers used to imperative programming style to switch to functional or even reactive programming? I show the same example implemented in multiple styles to highlight the advantages one would get by going from imperative to object-oriented, functional, lazy evaluation, promise-based, event emitters, and finally reactive approach. The short examples are in JavaScript - the most widely used functional language today.

  • 7,152

More from Gleb Bahmutov