Dr. Gleb Bahmutov PhD

Kensho Boston / NYC

Coding in JS

We use JS because it is everywhere

Changing how JS developers program will have a huge impact on the software quality

in the world

Kensho - instant financial analytics

Kensho - instant financial analytics

Harvard Sq, Cambridge MA

1 WTC New York

Problem: code complexity is quickly growing

number of engineers

lifetime of the company

We need JavaScript to

  • be as clear as writing an English sentence (or clearer)
  • produce expected result
  • be simple to test
  • assemble complex logic from very simple parts

Use FP to implement Reactive Programming patterns

FP in JS

  • small pure functions
  • compose complex logic from simple steps
  • semi-performant async with promises
  • simple(r) data with immutable structures

reactive programming

  • every user action is stream of events
  • every server action is stream of events
  • streams can be processed using standard operations

2 major parts

Dev team (otherwise smart people)

Everyone has their own path to JS

Everyone is different

  • 100% jQuery
  • 30% FP in JS
  • 25% OO in JS

! junior vs senior

  • 50% jQuery
  • 90% CSS
  • 5% OO in JS
  • 20% jQuery
  • 30% CSS
  • 75% OO in JS
  • 90% FP in JS

In theory

anyone

functional reactive programmer

In practice

anyone

functional reactive programmer

...

...

...

...

...

...

...

  • procedural
  • object-oriented
  • functional
  • abandon return values
  • immutable data
  • lazy evaluation
  • lazy async values
  • promises
  • event emitters
  • event emitters returning promises
  • chained event emitters
  • reactive streams

Here is my journey through the JS land

Show me the code!

 

Problem: given an array of numbers, multiply each number by a constant and print the result.

var numbers = [3, 1, 7];
var constant = 2;
// 6 2 14

procedural / imperative JS

var numbers = [3, 1, 7];
var constant = 2;
var k = 0;
for(k = 0; k < numbers.length; k += 1) {
  console.log(numbers[k] * constant);
}
// 6 2 14
  • "for" loop is a pain to debug and use
  • mixing data manipulation with printing
    • code is hard to reuse

Start splitting code into procedures

var numbers = [3, 1, 7];
var constant = 2;

function mul(a, b) {
  return a * b;
}

function processNumber(n, constant) {
  console.log(mul(n, constant));
}

for(k = 0; k < numbers.length; k += 1) {
  processNumber(numbers[k], constant);
}
// 6 2 14

Make the code boring

Boring code does not surprise you

// for loops WILL surprise you
for(k = 0; k < numbers.length; k += 1) {
  processNumber(numbers[k], constant);
}

Boring code does not surprise you

// using "this" WILL surprise you
numbers.forEach(function process(n) {
  return this.mul(n, 2);
});

Boring code does not surprise you

// using non-local variables WILL surprise you
constant = 2;
...
numbers.forEach(function process(n) {
  return mul(n, constant);
});

Pure functions are boring

function mul(a, b) {
  return a * b;
}
function print(n) {
  console.log(n);
}

Boring functions make new boring functions

var mulBy2 = mul.bind(null, 2);
var mul2by3 = mul.bind(null, 2, 3);
var print = console.log.bind(console, 'RESULT');
function mul(a, b) {
  return a * b;
}
print(mulBy2(10));
// RESULT 20
print(mul2by3(), mul2by3());
// RESULT 6 6
var numbers = [3, 1, 7];
var constant = 2;
function processNumber(n, constant) { ... }

// use built-in Array.prototype.forEach
numbers.forEach(function (n) {
  processNumber(n, constant);
});
// 6 2 14

no more for-loop !

Object-Oriented EcmaScript5

Object-oriented + functional

var numbers = [3, 1, 7];
var constant = 2;
function mul(a, b) {
  return a * b;
}
function print(n) {
  console.log(n);
}
function mulBy = mul.bind(null, constant);
numbers.map(mulBy).forEach(print);

// 6 2 14
  • clear "multiply then print" semantics

A lot of our code

manipulates collections of data items

// OO (built-in)
Array.prototype.map(cb)

// functional (user space)
Library.map(array, cb)

// plus
Library.chunk(array, cb);
Library.drop(array, cb);
...

2 of the top 3 most dependent upon NPM projects are FP utility libraries: underscorelodash

var _ = require('lodash');
var byConstant = _.partial(mul, constant);
_(numbers)
  .map(byConstant)
  .forEach(print);
// 6 2 14

Work with arrays

_({ John: 3, Jim: 10 })
  .map(byConstant)
  .forEach(print);
// 6 20

Treat objects same as collections

Argument order

var mulBy = ...

// OO (built-in)
Array.prototype.map(mulBy)

// lodash / underscore
_.map(array, mulBy)
// Ramda
R.map(mulBy, array)
var R = require('ramda');
var byConstant = R.partial(R.mul, constant);
var mapByConstant = R.partial(R.map, byConstant);
var printEach = R.forEach(print);
var algorithm = R.compose(printEach, mapByConstant);

algorithm(numbers);
// 6 2 14

1. compose algorithm

2. apply to data

3. profit!!!

// R.mul = function (a, b) { ... }
var byConstant = R.partial(R.mul, constant);
// same as
var byConstant = R.mul(constant);

Partial application => curry

var R = require('ramda');
var mapByConstant = R.map(R.mul(constant));
var algorithm = R.compose(
  R.forEach(print), 
  mapByConstant
);

algorithm(numbers);
// 6 2 14

No custom code necessary

OO

"Smart" objects

Pass data around

FP

"Dumb" objects

"Smart" pipeline

Pass code (functions) around

new NumberMultiplier()
    .add(3, 1, 7)
    .multiply(2)
    .print();
_([3, 1, 7])
    .map(_.partial(mul, 2))
    .forEach(print);

Object-oriented VS functional

It is better to have 100 functions operate on one data structure than to have 10 functions operate on 10 data structures."

                                  - Alan J. Perlis

new NumberMultiplier()
    .add(3, 1, 7)
    .multiply(2)
    .print();
_([3, 1, 7])
    .map(_.partial(mul, 2))
    .forEach(print);

Mutable data

var numbers = [...];
numbers.forEach(processNumber);
// is the list "numbers" unchanged?
function processNumber(x, k, array) {
    array[1] = 10000;
    print(x);
}

Mutable data makes code hard to understand and modify

Immutable data structures

'use strict';
var immutable = require('seamless-immutable');
var byConstant = _.partial(mul, constant);
immutable(numbers)
  .map(byConstant)
  .forEach(function (x, k, array) {
    array[1] = 10000; // throws an Error
    print(x);
  });

Efficient immutable JS libraries: seamless-immutablemori

Say "Goodbye" to old friends

Tell return values Bye-bye

// imperative code
var sum = add(2, 3);
// can use sum right away!

var multiplied = [1, 2, 3].map(mul);
// can use multiplied array right away!
var _ = require('lodash');
var byConstant = _.partial(mul, constant);
_(numbers)
  .map(byConstant)
  .forEach(print);
// we have not returned any values!

Tell middle functions Bye-bye

var byConstant = _.partial(mul, constant);
_(numbers)
  .map(function (x) {
    return byConstant(x);
  })
  .forEach(function (x) {
    print(x);
  });
_(numbers)
  .map(byConstant)
  .forEach(print);
// no middle functions

Tell deep equality checks Bye-bye

var result = process(numbers);
if (deepEqual(result, numbers)) {
  ...
}
var result = process(immutableNumbers);
if (result === immutableNumbers) {
  ...
}

What if some functions are async?

// Does not work
numbers.map(function (x, done) {
  mul(x, function cb(result) {
    done(result);
  })
}).forEach(function (value) {
  print(value, function cb() {
    done():
  })
});

Need async callbacks and steps

numbers.map(asyncFn)
  // async step
  .forEach(asyncFn);

Promises

var Q = require('q');
Q(...)
    .then(function onSuccess(result) { ... },
          function onFailure(error) { ... })
    .then(...);

A single async action with a single result or error

var Q = require('q');
function foo() {
  var defer = Q.defer();
  ... 
    defer.resolve(returnValue);
  ...
  return defer.promise;
}
foo()
    .then(bar)
    .then(baz);
// functions foo, bar, baz return a promise

Each step in the promise chain can be asynchronous and return a promise!

var sleepSecond = _.partial(Q.delay, 1000);

var pauseMulAndPrint = function (n) {
  return function () {
    return sleepSecond()
        .then(_.partial(byConstant, n))
        .then(print);
  };
};

numbers.map(pauseMulAndPrint)
  .reduce(Q.when, Q())
  .done();
// ... 6 ... 2 ... 14

Sleep, mul then print using promises

Promises handle single event very well

Extra complexity trying to handle sequence of events

var sleepSecond = _.partial(Q.delay, 1000);

var pauseMulAndPrint = function (n) {
  return function () {
    return sleepSecond()
        .then(_.partial(byConstant, n))
        .then(print);
  };
};

numbers.map(pauseMulAndPrint)
  .reduce(Q.when, Q())
  .done();
// ... 6 ... 2 ... 14

Promises handle single event very well

Extra complexity trying to handle sequence of events

var sleepSecond = _.partial(Q.delay, 1000);

var pauseMulAndPrint = function (n) {
  return function () {
    return sleepSecond()
        .then(_.partial(byConstant, n))
        .then(print);
  };
};

numbers.map(pauseMulAndPrint)
  // returned list is list of promises!
  .reduce(Q.when, Q())
  .done();
// ... 6 ... 2 ... 14

Constructing a promise for each number!

var sleepSecond = _.partial(Q.delay, 1000);

var pauseMulAndPrint = function (n) {
  return function () {
    return sleepSecond()
        .then(_.partial(byConstantAsync, n))
        .then(print);
  };
};

Need to chain async actions without boilerplate per item

Event Emitters

  • handle repeated events

  • decouple the event source from an action

Widely used in browser code

  • UI events 
  • Window communication 
  • Websockets communication 
$(el).on('click', action)

window.onmessage = action

socket.on('message', action)

We can extend the same approach to any event generator (like number sequence)

// action
var events = require('events');
var numberEmitter = new events.EventEmitter();
numberEmitter.on('number', 
    _.compose(print, byConstant)
);

Single code pipeline processes all numbers

// async event source
lazy(numbers)
  .async(1000)
  .each(
      _.bind(numberEmitter.emit, numberEmitter, 'number')
  );
// prints 6, 2 14 with 1 second intervals

Process number with 1 second pauses

function source(generate) {
  var eventEmitter = new events.EventEmitter();
  generate(
    _.bind(eventEmitter.emit, eventEmitter, 'step')
  );

  return {
    on: function (cb) {
      eventEmitter.on('step', cb);
    }
  };
}

Decouple emit / on code

var generator = lazy(list).async(1000);
source(_.bind(generator.each, generator))
  .on(_.compose(print, byConstant));
// prints 6, 2 14 with 1 second intervals
var generator = lazy(list).async(1000);
var generatorFn = _.bind(generator.each, generator);
var actionFn = _.compose(print, byConstant);

source(generatorFn)
  .on(actionFn);

Just the user code

Are we overcomplicating things?

// sync event generation
// sync action
action(eventGenerator);
// any event generation
// async action
source(eventGenerator)
  .on(action);

Our emitter has `on`

  1. What is the semantic meaning of `on`?
  2. How do we connect multiple steps?
source(eventGenerator)
  .on(action);

Remember Array ES5 methods?

[3, 1, 7]
  .map(mulBy2)
  .forEach(print);

I want this!

source of events (numbers)
  .map(an async callback)
  .buffer(n, async callback)
  .forEach(another async callback);

Chained emitters

var stepEmitter = {

  map: function (cb) {
    var emitter = new events.EventEmitter();




    return _.extend(emitter, stepEmitter);
  },

  forEach: function (cb) {
    var emitter = new events.EventEmitter();




    return _.extend(emitter, stepEmitter);
  }
};

Chained emitters

var stepEmitter = {

  map: function (cb) {
    var emitter = new events.EventEmitter();
    this.on('step', function (value) {
      var mappedValue = cb(value);
      emitter.emit('step', mappedValue);
    });
    return _.extend(emitter, stepEmitter);
  },

  forEach: function (cb) {
    var emitter = new events.EventEmitter();
    this.on('step', function (value) {
      cb(value);
      emitter.emit('step', value);
    });
    return _.extend(emitter, stepEmitter);
  }
};

Chained emitters

var stepEmitter = {

  map: function (cb) {
    var emitter = new events.EventEmitter();
    this.on('step', function (value) {
      var mappedValue = cb(value);
      emitter.emit('step', mappedValue);
    });
    return _.extend(emitter, stepEmitter);
  },

  forEach: function (cb) {
    var emitter = new events.EventEmitter();
    this.on('step', function (value) {
      cb(value);
      emitter.emit('step', value);
    });
    return _.extend(emitter, stepEmitter);
  }
};
source of events
  .map(syncFn) 
    // next step async
  .forEach(syncFn); 
    // next step async

Chained emitters

Chained emitters + promises

var Q = require('q');
var stepEmitter = {
  map: function (cb) {
    var emitter = new events.EventEmitter();
    this.on('step', function (value) {
      Q.when(cb(value)).then(function (mappedValue) {
        emitter.emit('step', mappedValue);
      });
    });
    return _.extend(emitter, stepEmitter);
  },
  forEach: function (cb) {
    var emitter = new events.EventEmitter();
    this.on('step', function (value) {
      Q.when(cb(value)).then(function () {
        emitter.emit('step', value);
      });
    });
    return _.extend(emitter, stepEmitter);
  }
};
source of events (numbers)
  .map(asyncFn) 
    // next step async
  .forEach(asyncFn); 
    // next step async

Chained emitters + promises

var stepEmitter = {
  // same map and forEach methods as above

  // returns a new step emitter that accumulates N items
  // emits the entire array with N items as single argument
  buffer: function (n) {
    var received = [];
    var emitter = new events.EventEmitter();
    this.on('step', function (value) {
      received.push(value);
      if (received.length === n) {
        emitter.emit('step', received);
        received = [];
      }
    });
    return _.extend(emitter, stepEmitter);
  }
};
source(numbers)
  .map(byConstant)
  .buffer(3)
  .forEach(print);
// sleeps 3 seconds then prints [6, 2, 14]
source(numbers)
  .map(byConstant) // data transform
  .buffer(3)       // sequence control
  .forEach(print); // data transform

We implemented 2 types

of operations

on a sequence of events

Need extra methods to control the sequence (stream) of events

.map, .filter
// vs
.buffer

Reactive programming

  • Everything is a source of asynchronous events.
  • Your code is a pipeline reacting to one or more input streams and outputting another stream of events

Reactive JS libraries

  • collection processing methods (map, filter, ...)
  • stream control methods (buffer, zip, flatMapLatest, ...)

Use a reactive FP library

var Rx = require('rx');

var timeEvents = Rx.Observable
  .interval(1000)
  .timeInterval(); // stream 1
var numberEvents = Rx.Observable
  .fromArray(numbers); // stream 2
Rx.Observable.zip(timeEvents, numberEvents, 
                  function pickValue(t, n) { return n; })
  .map(byConstant)
  .subscribe(print); // stream 3

// prints 6 2 14 with 1 second intervals

User click events

var Rx = require('rx');

var clickEvents = Rx.Observable
  .fromEvent(document.querySelector('#btn'), 'click')

var numberEvents = Rx.Observable
  .fromArray(numbers);

Rx.Observable.zip(clickEvents, numberEvents, 
                  function pickValue(whatever, n) { 
                    return n; 
                  })
  .map(byConstant)
  .subscribe(print);
// prints a number on each button click
function searchWikipedia (term) {
    return $.ajax({
        url: 'http://en.wikipedia.org/w/api.php',
        dataType: 'jsonp',
        data: {
            action: 'opensearch',
            format: 'json',
            search: term
        }
    }).promise();
}
var keyups = Rx.Observable.fromEvent($('#input'), 'keyup')
    .map(function (e) { return e.target.value; })
    .filter(function (text) { return text.length > 2; });

keyups.throttle(500)
    .distinctUntilChanged()
    .flatMapLatest(searchWikipedia)
    .subscribe(function (data) {
        // display results
    });

Autocomplete

Summary

  • Write small functions

  • Write small pure functions

  • Handle single async events using promises

  • Handle multiple async event streams using reactive

From procedural to RFP

Q: Which step causes difficulties?

A: For me it was abandoning the return values, others had problem with promises

Q: How to nudge developers along the journey?

A: Having everyone interested train together

Kensho progress report

object-oriented        **             2 /10
functional             *********      9 /10
immutable data         **             2 /10
lazy evaluation        *              1 /10
promises               *********      9 /10
event emitters         *******        7 /10
reactive streams       *****          5 /10

Future is bright

All examples shown use today's ES5 standard


  • ES6 (EcmaScript2015) will make some pieces simpler
    • arrow notation
    • block-scoping and constants
    • spread operator
    • native Promises
    • binding context in callbacks

Journey from procedural to reactive JavaScript with stops - FrontEnd Camp

By Gleb Bahmutov

Journey from procedural to reactive JavaScript with stops - FrontEnd Camp

How do we convince / train developers used to imperative programming style to switch to functional or even reactive programming? I show the same example implemented in multiple styles to highlight the advantages one would get by going from imperative to object-oriented, functional, lazy evaluation, promise-based, event emitters, and finally reactive approach. The short examples are in JavaScript - the most widely used functional language today.

  • 2,836
Loading comments...

More from Gleb Bahmutov