Journey from procedural to reactive JavaScript with stops

Dr. Gleb Bahmutov PhD

Kensho Boston / NYC

• Personal site http://glebbahmutov.com/
• Blog http://glebbahmutov.com/blog/
• Twitter @bahmutov

WAIT,

?

Language where equality == looks like this?

source: Don't make JavaScript equality look worse than it is

Language where < looks like this?

source: Don't make JavaScript equality look worse than it is

why
js
matters

JavaScript - not just Netscape 

• browser
• server
• mobile
• embedded

source: StackOverflow

source: StackOverflow

clojure 19
haskell 21
ocaml   11
scala   105
elm     0
f#      16
erlang  21
idris   0
-----------
        193

source: http://careers.stackoverflow.com/jobs?searchTerm=<language>  in February 2015

similar results from: http://www.sitepoint.com/whats-best-programming-language-learn-2015/

similar results when searching by company: http://careers.stackoverflow.com/companies?searchTerm=<language>

source: really? you don't know who these are?

120k tweets per hour

Changing how JS developers program will have a huge impact on the software quality

in the world

Kensho financial analytics

Problem: code complexity is quickly growing

We need JavaScript to

• be as clear as writing an English sentence (or clearer)
• produce expected result
• be simple to test
• assemble complex logic from very simple parts

Use FP to implement Reactive Programming patterns

• small pure functions
• compose complex logic from simple steps
• semi-performant async with promises
• simple(r) data with immutable structures

• every user action is stream of events
• every server action is stream of events
• streams can be processed using standard operations

2 major parts

Dev team (otherwise smart people)

Everyone has their own path to JS

Everyone is different

! junior vs senior

In theory

anyone

functional reactive programmer

Here is my journey through the JS land

• procedural
• object-oriented
• functional
• abandon return values
• immutable data
• lazy evaluation
• lazy async values
• promises
• event emitters
• event emitters returning promises
• chained event emitters
• reactive streams

Possible shortcuts

• procedural
• object-oriented
• functional
• abandon return values
• immutable data
• lazy evaluation
• lazy async values
• promises
• event emitters
• event emitters returning promises
• chained event emitters
• reactive streams

procedural / imperative JS

var numbers = [3, 1, 7];
var constant = 2;
var k = 0;
for(k = 0; k < numbers.length; k += 1) {
  console.log(numbers[k] * constant);
}
// 6 2 14

• "for" loop is a pain to debug and use
• mixing data manipulation with printing​
  • code is hard to reuse

var numbers = [3, 1, 7];
var constant = 2;
var k = 0;
for(k = 0; k < numbers.length; k += 1) {
  console.log(numbers[k] * constant);
}
// 6 2 14

Programming without for loops

Start splitting code into procedures

var numbers = [3, 1, 7];
var constant = 2;

function mul(a, b) {
  return a * b;
}

function processNumber(n, constant) {
  console.log(mul(n, constant));
}

for(k = 0; k < numbers.length; k += 1) {
  processNumber(numbers[k], constant);
}
// 6 2 14

Make the code boring

Boring code does not surprise you

// for loops WILL surprise you
for(k = 0; k < numbers.length; k += 1) {
  processNumber(numbers[k], constant);
}

Boring code does not surprise you

// using "this" WILL surprise you
numbers.forEach(function process(n) {
  return this.mul(n, 2);
});

Boring code does not surprise you

// using non-local variables WILL surprise you
constant = 2;
...
numbers.forEach(function process(n) {
  return mul(n, constant);
});

Pure functions are boring

function mul(a, b) {
  return a * b;
}
function print(n) {
  console.log(n);
}

• output depends ONLY on the inputs
  • same result every time
• no surprises
• pure against set of unit tests
  • Test if a function is pure

Boring functions make new boring functions

function mul(a, b) {
  return a * b;
}
var mulBy2 = mul.bind(null, 2);
var mul2by3 = mul.bind(null, 2, 3);
var print = console.log.bind(console, 'RESULT');

print(mulBy2(10));
// RESULT 20
print(mul2by3(), mul2by3());
// RESULT 6 6

Array.map

Produces new array

Passes each number through

a callback function

Array.forEach

Returns nothing

Side effects

Array.filter

Produces new array

Passes only items for which the callback returns truthy value

Object-Oriented EcmaScript5

Array.reduce

List -> single value

Reigns supreme

Built-in .forEach

var numbers = [3, 1, 7];
var constant = 2;

// use built-in Array.prototype.forEach
numbers.forEach(function (n) {
  processNumber(n, constant);
});
// 6 2 14

• no more for-loop !

Object-oriented way

var numbers = [3, 1, 7];
var constant = 2;

function mul(a, b) {
  return a * b;
}
function print(n) {
  console.log(n);
}
numbers.map(function (n) {
  return mul(n, constant);
}).forEach(print);

// 6 2 14

• clear "multiply then print" semantics

A lot of our code

manipulates collections of data items

// OO (built-in)
Array.prototype.something(x, y)

// functional (user space)
Library.something(array, x, y)
Library.another(array, k);

2 of the top 3 most dependent upon NPM projects are FP utility libraries: underscore, lodash

var _ = require('lodash');
var byConstant = _.partial(mul, constant);
_(numbers)
  .map(byConstant)
  .forEach(print);
// 6 2 14

var _ = require('lodash');
_.forEach(
    _.map(numbers, byConstant), 
    print);
// 6 2 14

Argument order

// OO (built-in)
Array.prototype.something(x, y)

// lodash / underscore
Library.something(array, x, y)

// Ramda
Library.something(x, y, array)

Put callback first for elegance

var R = require('ramda');
var mapByConstant = R.map(byConstant);
var printEach = R.forEach(print);
var algorithm = R.compose(printEach, mapByConstant);

algorithm(numbers);
// 6 2 14

1. compose algorithm

2. apply to data

3. profit!!!

_({ John: 3, Jim: 10 })
  .map(byConstant)
  .forEach(print);
// 6 20

Treat objects same as collections

object iterators

OO

"Smart" objects

Pass data around

new NumberMultiplier()
    .add(3, 1, 7)
    .multiply(2)
    .print();

new NumberMultiplier()
    .add(3, 1, 7)
    .multiply(2)
    .print();

_([3, 1, 7])
    .map(_.partial(mul, 2))
    .forEach(print);

It is better to have 100 functions operate on one data structure than to have 10 functions operate on 10 data structures."

                                  - Alan J. Perlis

Skip: Transducers

composable algorithmic transformations

Read the intro

Say "Goodbye" to old friends

Tell return values Bye-bye

// imperative code
var sum = add(2, 3);
// can use sum right away!

var multiplied = [1, 2, 3].map(mul);
// can use multiplied array right away!

Tell middle functions Bye-bye

var byConstant = _.partial(mul, constant);
_(numbers)
  .map(function (x) {
    return byConstant(x);
  })
  .forEach(function (x) {
    print(x);
  });

Point-free is not pointless

Tell side-effects Bye-bye

var byConstant = _.partial(mul, constant);
_(numbers)
  .map(byConstant)
  .forEach(function (x, k, array) {
    array[1] = 10000;
    print(x);
  });

with immutable data structures

Tell side-effects Bye-bye

'use strict';
var immutable = require('seamless-immutable');
var byConstant = _.partial(mul, constant);
immutable(numbers)
  .map(byConstant)
  .forEach(function (x, k, array) {
    array[1] = 10000; // throws an Error
    print(x);
  });

with immutable data structures

Efficient immutable JS library mori

Lazy evaluation

var _ = require('lodash');
// lodash v3.x.x

var byConstant = _.partial(mul, constant);
// avoid intermediate arrays
_(numbers)
  .map(byConstant)
  .take(2)
  .forEach(print);
  .value();
// 6
// 2

Lazy async eval

var lazy = require('lazy.js');
lazy(numbers)
  .async(1000) // 1000ms = 1 sec
  .map(byConstant)
  .each(print);
// 2
// 1 second later
// 6
// 1 second later
// 14

Promises

var Q = require('q');
Q(...)
    .then(function onSuccess(result) { ... },
          function onFailure(error) { ... })
    .then(...);

A single async action with a single result or error

Promises

var Q = require('q');
function asyncMul(a, b) {
  return Q(a * b).delay(1000);
}
var byConstantAsync = _.partial(asyncMul, constant);
var promiseToMulNumber = function (n) {
  return byConstantAsync(n);
};

Q.all(numbers.map(promiseToMulNumber))
  .then(print)
  .done();
// sleeps 1 second
// then prints [6, 2, 14]

Promise chain

function asyncMul(a, b) {
  return Q(mul(a, b))
}
var sleep1 = _.partial(Q.delay, 1000);

Q()
  .then(sleep1)
  .then(_.partial(byConstantAsync, numbers[0]))
  .then(print)
  .then(sleep1)
  .then(_.partial(byConstantAsync, numbers[1]))
  .then(print)
  .then(sleep1)
  .then(_.partial(byConstantAsync, numbers[2]))
  .then(print)
  .done();
// ... 6 ... 2 ... 14

Promise chain

function asyncMul(a, b) {
  return Q(mul(a, b))
}
var sleep1 = _.partial(Q.delay, 1000);

var mulAndPrint = function (n) {
  return function () {
    return sleep1()
        .then(_.partial(byConstantAsync, n))
        .then(print);
  };
};

numbers.map(mulAndPrint)
  .reduce(Q.when, Q())
  .done();
// ... 6 ... 2 ... 14

Extra complexity

function asyncMul(a, b) {
  return Q(mul(a, b))
}
var sleep1 = _.partial(Q.delay, 1000);

var mulAndPrint = function (n) {
  return function () {
    return sleep1()
        .then(_.partial(byConstantAsync, n))
        .then(print);
  };
};

numbers.map(mulAndPrint)
  .reduce(Q.when, Q())
  .done();
// ... 6 ... 2 ... 14

Promises handle single event

Extra complexity trying to handle sequence of events

function asyncMul(a, b) {
  return Q(mul(a, b))
}
var sleep1 = _.partial(Q.delay, 1000);

var mulAndPrint = function (n) {
  return function () {
    return sleep1()
        .then(_.partial(byConstantAsync, n))
        .then(print);
  };
};

numbers.map(mulAndPrint)
  .reduce(Q.when, Q())
  .done();
// ... 6 ... 2 ... 14

Event Emitters

• handle repeated events

• decouple the event source

from an action

// action
var events = require('events');
var numberEmitter = new events.EventEmitter();
numberEmitter.on('number', 
    _.compose(print, byConstant)
);

// async event source
lazy(numbers)
  .async(1000)
  .each(
      _.bind(numberEmitter.emit, numberEmitter, 'number')
  );
// prints 6, 2 14 with 1 second intervals

// action
var events = require('events');
var numberEmitter = new events.EventEmitter();
numberEmitter.on('number', 
    _.compose(print, byConstant)
);

// async event source
lazy(numbers)
  .async(1000)
  .each(
      _.bind(numberEmitter.emit, numberEmitter, 'number')
  );
// prints 6, 2 14 with 1 second intervals

Everything but marked lines is boilterplate

function source(list) {
  var eventEmitter = new events.EventEmitter();
  lazy(list)
    .async(1000)
    .each(
      _.bind(eventEmitter.emit, eventEmitter, 'step')
    );

  return {
    on: function (cb) {
      eventEmitter.on('step', cb);
    }
  };
}
source(numbers)
  .on(_.compose(print, byConstant));
// prints 6, 2 14 with 1 second intervals

Decouple action

function source(generate) {
  var eventEmitter = new events.EventEmitter();
  generate(
    _.bind(eventEmitter.emit, eventEmitter, 'step')
  );

  return {
    on: function (cb) {
      eventEmitter.on('step', cb);
    }
  };
}

var generator = lazy(list).async(1000);
source(_.bind(generator.each, generator))
  .on(_.compose(print, byConstant));
// prints 6, 2 14 with 1 second intervals

Decouple generator

var generator = lazy(list).async(1000);
source(_.bind(generator.each, generator))
  .on(_.compose(print, byConstant));

Just the user code

Widely used in browser code

• UI events 
• Window communication 
• Websockets communication 

$(el).on('click', action)

window.onmessage = action

socket.on('message', action)

Are we overcomplicating things?

// sync event generation
// sync action
action(eventGenerator);

Our emitter has `on`

1. What is the semantic meaning of `on`?
2. How do we connect multiple steps?

source(eventGenerator)
  .on(action);

Remember Array ES5 methods?

[3, 1, 7]
  .map(mulBy2)
  .forEach(print);

I want this!

source of events
  .map(an async callback)
  .forEach(another async callback);

Chained emitters

var stepEmitter = {

  map: function (cb) {
    var emitter = new events.EventEmitter();
    this.on('step', function (value) {
      var mappedValue = cb(value);
      emitter.emit('step', mappedValue);
    });
    return _.extend(emitter, stepEmitter);
  },

  forEach: function (cb) {
    var emitter = new events.EventEmitter();
    this.on('step', function (value) {
      cb(value);
      emitter.emit('step', value);
    });
    return _.extend(emitter, stepEmitter);
  }
};

// returns stepEmitter = event emitter + stepEmitter methods
function source(list) {
  var eventEmitter = new events.EventEmitter();
  lazy(list)
    .async(1000)
    .each(_.bind(eventEmitter.emit, eventEmitter, 'step'));

  return _.extend(eventEmitter, stepEmitter);
}

// each step returns s new stepEmitter
source(numbers)
  .map(byConstant)
  .forEach(print);
// prints 6, 2 14 with 1 second intervals

var stepEmitter = {
  // same map and forEach methods as above

  // returns a new step emitter that accumulates N items
  // emits the entire array with N items as single argument
  buffer: function (n) {
    var received = [];
    var emitter = new events.EventEmitter();
    this.on('step', function (value) {
      received.push(value);
      if (received.length === n) {
        emitter.emit('step', received);
        received = [];
      }
    });
    return _.extend(emitter, stepEmitter);
  }
};
source(numbers)
  .map(byConstant)
  .buffer(3)
  .forEach(print);
// sleeps 3 seconds then prints [6, 2, 14]

source(numbers)
  .map(byConstant) // data transform
  .buffer(3)       // sequence control
  .forEach(print); // data transform

Need extra methods to control the sequence

.map, .filter
// vs
.buffer

Skip: Node streams

Take a look at Highland.js

Each high level "event" can be a sequence of low-level built-in "events"

Reactive programming

• Everything is a source of asynchronous events.
• Your code is a pipeline reacting to one or more input streams and outputting another stream

Use a reactive FP library

var Rx = require('rx');

var timeEvents = Rx.Observable
  .interval(1000)
  .timeInterval(); // stream 1

var numberEvents = Rx.Observable
  .fromArray(numbers); // stream 2

Rx.Observable.zip(timeEvents, numberEvents, 
                  function pickValue(t, n) { return n; })
  .map(byConstant)
  .subscribe(print); // stream 3

// prints 6 2 14 with 1 second intervals

Decoupled time from numbers

var timeEvents = Rx.Observable
  .interval(1000)
  .timeInterval();

var numberEvents = Rx.Observable
  .fromArray(numbers);

Rx.Observable.zip(timeEvents, numberEvents, 
                  function pickValue(t, n) { 
                    return n; 
                  });

User events

var clickEvents = Rx.Observable
  .fromEvent(document.querySelector('#btn'), 'click')

var numberEvents = Rx.Observable
  .fromArray(numbers);

Rx.Observable.zip(clickEvents, numberEvents, 
                  function pickValue(whatever, n) { 
                    return n; 
                  });

function searchWikipedia (term) {
    return $.ajax({
        url: 'http://en.wikipedia.org/w/api.php',
        dataType: 'jsonp',
        data: {
            action: 'opensearch',
            format: 'json',
            search: term
        }
    }).promise();
}
var keyups = Rx.Observable.fromEvent($('#input'), 'keyup')
    .map(function (e) {
        return e.target.value;
    })
    .filter(function (text) {
        return text.length > 2;
    });

keyups.throttle(500)
    .distinctUntilChanged()
    .flatMapLatest(searchWikipedia)
    .subscribe(function (data) {
        // display results
    });

Reactive JS libraries

• collection processing methods (map, filter, ...)
• stream control methods (zip, flatMapLatest, ...)

RxJs, BaconJS, cycle

From procedural to RFP

• Reuse small pure functions
• Single async event using promises
• Async event streams using reactive

Kensho progress report

object-oriented        **             2 /10

functional             ********       8 /10

immutable data         **             2 /10

lazy evaluation        *              1 /10

promises               *********      9 /10

event emitters         *******        7 /10

reactive streams       *              1 /10

Future is bright

• All examples shown use today's ES5 standard
• ES6 (EcmaScript2015) will make some pieces simpler
  • arrow notation
  • block-scoping
  • spread operator
  • native Promises
  • binding context in callbacks

Dr. Gleb Bahmutov PhD

Kensho

• Personal site http://glebbahmutov.com/
• Blog http://glebbahmutov.com/blog/
• Twitter @bahmutov

• this presentation at slides.com/bahmutov/javascript-journey
• this presentation as a blog post http://glebbahmutov.com/blog/journey-from-procedural-to-reactive-javascript-with-stops/