Journey from procedural to reactive JavaScript with stops

Dr. Gleb Bahmutov PhD

Cypress.io

Coding in JS

source: Improved Minesweeper

JavaScript == table

source: Don't Make Javascript Equality Look Worse Than It Is 

JavaScript < table

source: Don't Make Javascript Equality Look Worse Than It Is 

We use JS because it is everywhere

  • browser

  • server

  • mobile

  • robots

cyclon.js

source: careers.stackoverflow.com

JS package growth

source: modulecounts.com

JS package growth

source: modulecounts.com

Changing how JS developers program will have a huge impact on the software quality in the world

Problem: code complexity is quickly growing

number of engineers

lifetime of the company

We need JavaScript to

  • be as clear as writing an English sentence (or clearer)
  • produce expected result
  • be simple to test
  • assemble complex logic from very simple parts

Use FP to implement Reactive Programming patterns

reactivex.io/tutorials.html

js books

FP in JS

  • small pure functions
  • compose complex logic from simple steps
  • semi-performant async with promises
  • simple(r) data with immutable structures

reactive programming

  • every user action is stream of events
  • every server action is stream of events
  • streams can be processed using standard operations

2 major parts

Dev team (otherwise smart people)

Everyone has their own path to JS

Everyone is different

  • 100% jQuery
  • 30% FP in JS
  • 25% OO in JS

! junior vs senior

  • 50% jQuery
  • 90% CSS
  • 5% OO in JS
  • 20% jQuery
  • 30% CSS
  • 75% OO in JS
  • 90% FP in JS

In theory

anyone

functional reactive programmer

In practice

anyone

functional reactive programmer

...

...

...

...

...

...

...

  • procedural
  • object-oriented
  • functional
  • abandon return values
  • immutable data
  • lazy evaluation
  • lazy async values
  • promises
  • event emitters
  • event emitters returning promises
  • chained event emitters
  • reactive streams

Here is my journey through the JS land

Show me the code!

 

Problem: given an array of numbers, multiply each number by a constant and print the result.

var numbers = [3, 1, 7];
var constant = 2;
// 6 2 14

more than 20 implementations at 

github.com/bahmutov/javascript-journey

github.com/bahmutov/javascript-journey

procedural / imperative JS

var numbers = [3, 1, 7];
var constant = 2;
var k = 0;
for(k = 0; k < numbers.length; k += 1) {
  console.log(numbers[k] * constant);
}
// 6 2 14

  • "for" loop is a pain to debug and use

  • mixing data manipulation with printing​

    • code is hard to reuse

https://glebbahmutov.com/blog/programming-without-for-loops/

 

Start splitting code into procedures

var numbers = [3, 1, 7];
var constant = 2;
function mul(a, b) {
  return a * b;
}
function print(x) {
  console.log(x);
}
function processNumber(n) {
  print(mul(n, constant));
}
for(k = 0; k < numbers.length; k += 1) {
  processNumber(numbers[k]);
}
// 6 2 14

reusable simple functions

complex logic

iteration

Our goal

make the code boring

Boring code does not surprise you

// using "this" WILL surprise you
numbers.forEach(function (x) {
  this.doSomething(x);
  // ERROR!
});
var self = this;
numbers.forEach(function (x) {
  self.doSomething(x);
});

Boring code does not surprise you

// using "this" in your objects
// WILL surprise someone else
numbers.forEach(console.log);
numbers.forEach(console.log.bind(console));
// works under Node
// in Chrome
// Uncaught TypeError: Illegal invocation

Boring code does not surprise you

// for loops WILL surprise you
for(k = 0; k < numbers.length; k += 1) {
  processNumber(numbers[k]);
}

common problems: off by 1 errors, iterator variable, changing array size

Boring code does not surprise you

// using non-local variables WILL surprise you
constant = 2;
...
numbers.forEach(function process(n) {
  return mul(n, constant);
});

Pure functions are really boring

function mul(a, b) {
  return a * b;
}
function print(n) {
  console.log(n);
}

Boring functions make new boring functions using fn.bind

var mulBy2 = mul.bind(null, 2);
mulBy2(5); // 10

Binding vs partial application

function mul(a, b) {
  return a * b;
}

var mul2by3 = mul.bind(null, 2, 3);
mul2by3(); // 6
var mul2By10 = mulBy2.bind(null, 10);
mul2By10('extra', 'args are', 'ignored'); // 20

Partial application

function mul(a, b) {
  return a * b;
}
var by2 = mul.bind(null, 2);
// same as
var _ = require('lodash');
var by2 = _.partial(mul, 2);

_.partial, _.partialRight, spots, obind, heroin

Boring code is nice, but I want encapsulated logic, data access, inheritance!

function NumberMultiplier() {}

NumberMultiplier.prototype.setNumbers = function (numbers) {
    this.numbers = numbers;
    return this;
};
NumberMultiplier.prototype.multiply = function (constant) {
    for (var k = 0; k < this.numbers.length; k += 1) {
      this.numbers[k] = constant * this.numbers[k];
    }
    return this;
};
NumberMultiplier.prototype.print = function () {
    console.log(this.numbers);
};

Object-Oriented JavaScript

Object-Oriented JavaScript

// using NEW keyword
new NumberMultiplier()
  .setNumbers(numbers)
  .multiply(constant)
  .print();
// [ 6, 2, 14 ]

Object-Oriented JavaScript: 3 parts

function NumberMultiplier() {}

NumberMultiplier.prototype.setNumbers = function() {}
NumberMultiplier.prototype.multiply = function() {}
NumberMultiplier.prototype.print = function() {}

new NumberMultiplier();

constructor function

prototype

"new" keyword

// NumberMultiplier.prototype object
{
  setNumbers: function () { ... },
  multiply: function () { ... },
  print: function () { ... }
}
  /\
  || [[ prototype ]]
  ||
// instance object
{
  numbers: [3, 1, 7]
}

OO JavaScript is prototypical

Common Misconceptions About Inheritance in JavaScript

var NumberMultiplier = {
  setNumbers: function (numbers) {
    this.numbers = numbers; return this;
  },
  multiply: function (constant) {
    for (var k = 0; k < this.numbers.length; k += 1) {
      this.numbers[k] = constant * this.numbers[k];
    } return this;
  },
  print: function () { console.log(this.numbers); }
};

Prototype is a plain object!

javascript-journey source

var numberMultiplier = Object.create(NumberMultiplier);
numberMultiplier
  .setNumbers(numbers)
  .multiply(constant)
  .print();
// [ 6, 2, 14 ]

plain object

class NumberMultiplier {
  setNumbers(numbers) {
    this.numbers = numbers;
    return this;
  }
  multiply(constant) {
    for (var k = 0; k < this.numbers.length; k += 1) {
      this.numbers[k] = constant * this.numbers[k];
    }
    return this;
  }
  print() {
    console.log(this.numbers);
    return this;
  }
}
new NumberMultiplier()

ES6 classes: just don't

How to Fix the ES6 `class` keyword

JavaScript Arrays

typeof [1, 2] // "object"
Array.isArray([1, 2]) // true
Array.prototype
/*
  map
  forEach
  filter
  reduce
  ...
*/

Arrays: object-oriented + functional

var numbers = [3, 1, 7];
var constant = 2;
function mul(a, b) {
  return a * b;
}
function print(n) {
  console.log(n);
}
var mulBy = mul.bind(null, constant);
numbers
    .map(mulBy)
    .forEach(print);
// 6 2 14
  • clear "multiply then print" semantics

functional bits

Object-oriented methods

My advice

  1. Write small pure functions

  2. Adapt them to particular call sites

  3. Progress slowly through the code base

glebbahmutov.com/blog/point-free-programming-is-not-pointless/

Observation: a lot of our code

manipulates collections of data items

// OO (built-in)
Array.prototype.map(cb)


// functional (user space)
Library.map(array, cb)

// plus
Library.groupBy(array, cb);
Library.dropWhile(array, cb);
...

2 of the top 3 most dependent upon NPM projects are FP utility libraries: underscore, lodash 

var _ = require('lodash');
var byConstant = _.partial(mul, constant);
// _.forEach(array, cb);
// _.map(array, cb);

Work with arrays using Lodash

_.forEach(
  _.map(numbers, byConstant),
  print
);
// 6 2 14
_({ John: 3, Jim: 10 })
  .map(byConstant)
  .forEach(print);
// 6 20

Treat objects same as collections using lodash

object iterators

Argument order

var mulBy = ...

// OO (built-in)
Array.prototype.map(mulBy)

// lodash / underscore
_.map(array, mulBy)

glebbahmutov.com/blog/put-callback-first-for-elegance/

// Ramda
R.map(mulBy, array)
var R = require('ramda');
var byConstant = R.partial(R.mul, constant);

1. compose algorithm

2. apply to data

3. profit!!!

// R.forEach = function (callback, array)
var printEach = R.partial(R.forEach, print);
// R.map: function (callback, array)
var mapByConstant = R.partial(R.map, byConstant);

composition

printEach(mapByConstant(numbers));
// 6 2 14
var R = require('ramda');
var byConstant = R.partial(R.mul, constant);

1. compose algorithm

2. apply to data

3. profit!!!

// R.forEach = function (callback, array)
var printEach = R.partial(R.forEach, print);
var algorithm = R.compose(printEach, mapByConstant);

algorithm(numbers);
// 6 2 14
// R.map: function (callback, array)
var mapByConstant = R.partial(R.map, byConstant);
// fn( data )
var R = require('ramda');
var byConstant = R.partial(R.mul, constant);
// R.forEach = function (callback, array)
var printEach = R.partial(R.forEach, print);
var algorithm = R.compose(printEach, mapByConstant);

algorithm(numbers);
// 6 2 14
// R.map: function (callback, array)
var mapByConstant = R.partial(R.map, byConstant);

"fn(data)" code is simple to reason about and test

// fn( data )
// R.mul = function (a, b) { ... }
var byConstant = R.partial(R.mul, constant);

Partial application vs curry

// in Ramda this is same as simply
var byConstant = R.mul(constant);

// for your functions
var mul = R.curry(function (a, b) {
  return a * b; 
});
mul(5)(4); // 20

var R = require('ramda');
R.pipe(
  R.map(R.multiply(constant)),
  R.forEach(print)
)(numbers);
// 6 2 14

Curry: program with very little code

Requires careful function signature design

i.e. put information most likely to be known early as first argument

glebbahmutov.com/blog/user-friendly-api/

OO

"Smart" objects

Pass data around

FP

"Dumb" objects

"Smart" pipeline

Pass code (functions) around

new NumberMultiplier()
    .add(3, 1, 7)
    .multiply(2)
    .print();
_.forEach(
  _.map(numbers, byConstant),
  print
);

Object-oriented VS functional

It is better to have 100 functions operate on one data structure than to have 10 functions operate on 10 data structures."

                                  - Alan J. Perlis

new NumberMultiplier()
    .add(3, 1, 7)
    .multiply(2)
    .print();
_.forEach(
  _.map(numbers, byConstant),
  print
);

What else can JavaScript do for us?

Source of complexity: mutable data

var numbers = [...];
numbers.forEach(processNumber);
// is the list "numbers" unchanged?
function processNumber(x, k, array) {
    array[1] = 10000;
    print(x);
}

Mutable data makes code hard to understand and modify

Immutable data structures

'use strict';
var immutable = require('seamless-immutable');
var byConstant = _.partial(mul, constant);
immutable(numbers)
  .map(byConstant)
  .forEach(function (x, k, array) {
    array[1] = 10000; // throws an Error
    print(x);
  });

Efficient immutable JS libraries: seamless-immutable, mori

Immutable data with Redux pattern

function mul(a, b) { return a * b; }

function multiply(state, constant) {
  var byConstant = mul.bind(null, constant);
  return state.updateIn(['numbers'], function (ns) {
    return ns.map(byConstant);
  });
}

see the Redux example in javascript-journey

var immutable = require('immutable');
var initialState = immutable.Map({ 
  numbers: immutable.List.of([3, 1, 7])
});
var newState = multiply(initialState, constant);
// newState !== initialState

Arrays: great for synchronous actions

Efficient computation with transducers

Do as little as needed with lazy evaluation

Arrays: NOT SO great for asynchronous

or infinite data!

// Does NOT work
numbers.map(function (x, done) {
  mul(x, function cb(result) {
    done(result);
  })
}).forEach(function (value) {
  print(value, function cb() {
    done():
  })
});

How to handle async callbacks during iteration?

numbers
  .map(asyncFn)
  .forEach(asyncFn)

Promises

const p = new Promise((resolve, reject) => {
  ...
  resolve(42)
})
// after promise successfully completes
p.then(...)
 .catch(...)

Single async operation with a value or an error

Each callback can be synchronous or return a Promise

function sleep () {
  return new Promise(resolve => {
    setTimeout(resolve, 1000)
  })
}
p.then(sleep)
var sleepSecond = _.partial(Q.delay, 1000);

var pauseMulAndPrint = function (n) {
  return function () {
    return sleepSecond()
        .then(_.partial(byConstant, n))
        .then(print);
  };
};

Sleep, mul then print using promises

numbers.map(pauseMulAndPrint)
  .reduce(Q.when, Q())
  .done();
// ... 6 ... 2 ... 14

glebbahmutov.com/blog/chaining-promises/

Promises handle single event very well

Extra complexity trying to handle sequence of events

var sleepSecond = _.partial(Q.delay, 1000);

var pauseMulAndPrint = function (n) {
  return function () {
    return sleepSecond()
        .then(_.partial(byConstant, n))
        .then(print);
  };
};

numbers.map(pauseMulAndPrint)
  .reduce(Q.when, Q())
  .done();
// ... 6 ... 2 ... 14

Constructing a promise for each number :(

var sleepSecond = _.partial(Q.delay, 1000);

var pauseMulAndPrint = function (n) {
  return function () {
    return sleepSecond()
        .then(_.partial(byConstantAsync, n))
        .then(print);
  };
};

Promise is like a single cartridge

Improve upon promises: need to chain async actions without boilerplate per item

"shoot the same bullet more than once"

 

Event Emitters

  • handle repeated events

  • decouple the event source from an action

Widely used in browser code

$(el).on('click', action)

window.on('resize', cb)

socket.on('message', handle)

We can extend the same approach to any event generator (like number sequence)

// action
var events = require('events');
var numberEmitter = new events.EventEmitter();
numberEmitter.on('number', 
  _.compose(print, byConstant)
);

Single code pipeline

var k = 0;
var ref = setInterval(function () {
  numberEmitter.emit('number', numbers[k++]);
  if (k >= numbers.length) {
    clearInterval(ref);
  }
}, 1000);
// prints 6, 2 14 with 1 second intervals

Goal: process number with 1 second pauses

Our emitter has `on`

  1. What is the semantic meaning of `on`?
  2. How do we connect multiple steps?
numberEmitter
  .on('number', action);

Remember Array ES5 methods?

[3, 1, 7]
  .map(mulBy2)
  .forEach(print);

I want this!

source of number events
  .map(an async callback)
  .buffer(n, async callback)
  .forEach(another async callback);

Chained emitters

var stepEmitter = {

  map: function (cb) {
    var emitter = new events.EventEmitter();




    return _.extend(emitter, stepEmitter);
  },

  forEach: function (cb) {
    var emitter = new events.EventEmitter();




    return _.extend(emitter, stepEmitter);
  }
};

Chained emitters

var stepEmitter = {

  map: function (cb) {
    var emitter = new events.EventEmitter();
    this.on('step', function (value) {
      var mappedValue = cb(value);
      emitter.emit('step', mappedValue);
    });
    return _.extend(emitter, stepEmitter);
  },

  forEach: function (cb) {
    var emitter = new events.EventEmitter();
    this.on('step', function (value) {
      cb(value);
      emitter.emit('step', value);
    });
    return _.extend(emitter, stepEmitter);
  }
};

Chained emitters

var stepEmitter = {

  map: function (cb) {
    var emitter = new events.EventEmitter();
    this.on('step', function (value) {
      var mappedValue = cb(value);
      emitter.emit('step', mappedValue);
    });
    return _.extend(emitter, stepEmitter);
  },

  forEach: function (cb) {
    var emitter = new events.EventEmitter();
    this.on('step', function (value) {
      cb(value);
      emitter.emit('step', value);
    });
    return _.extend(emitter, stepEmitter);
  }
};

.map

emits cb(x)

.forEach

emits x

source of number events
  .map(syncFn) 
  .forEach(syncFn);

Chained emitters

So far: steps are async, but each callback function is synchronous!

Async callback support via promises

var Q = require('q');
var stepEmitter = {
  map: function (cb) {
    var emitter = new events.EventEmitter();
    this.on('step', function (value) {
      Q.when(cb(value)).then(function (mappedValue) {
        emitter.emit('step', mappedValue);
      });
    });
    return _.extend(emitter, stepEmitter);
  },

  forEach: function (cb) {
    var emitter = new events.EventEmitter();
    this.on('step', function (value) {
      Q.when(cb(value)).then(function () {
        emitter.emit('step', value);
      });
    });
    return _.extend(emitter, stepEmitter);
  }
};
source of number events
  .map(asyncFn) 
  .forEach(asyncFn);

Chained emitters + promises

async for the win!

var stepEmitter = {
  // same map and forEach methods as above

  // returns a new step emitter that accumulates N items
  // emits the entire array with N items as single argument
  buffer: function (n) {
    var received = [];
    var emitter = new events.EventEmitter();
    this.on('step', function (value) {
      received.push(value);
      if (received.length === n) {
        emitter.emit('step', received);
        received = [];
      }
    });
    return _.extend(emitter, stepEmitter);
  }
};
source(numbers)
  .map(byConstant)
  .buffer(3)
  .forEach(print);
// sleeps 3 seconds then prints [6, 2, 14]

What about buffering events?

source(numbers)
  .map(byConstant) // data transform
  .buffer(3)       // sequence control
  .forEach(print); // data transform

We implemented 2 types of operations on a sequence of events

Need extra methods to control the sequence (stream) of events

.map, .filter
// vs
.buffer

Reactive programming

  • Everything is a source of asynchronous events.
  • Your code is a pipeline reacting to one or more input streams and outputting another stream of events

Reactive JS libraries

  • collection processing methods (map, filter, ...)
  • stream control methods (buffer, zip, flatMapLatest, ...)

RxJs, BaconJS, cycle

Use a reactive FP library

var Rx = require('rx');

var timeEvents = Rx.Observable
  .interval(1000)
  .timeInterval(); // stream 1
var numberEvents = Rx.Observable
  .fromArray(numbers); // stream 2
Rx.Observable.zip(timeEvents, numberEvents, 
                  function pickValue(t, n) { return n; })
  .map(byConstant)
  .subscribe(print); // stream 3

// prints 6 2 14 with 1 second intervals

javascript-journey source

User click events

var Rx = require('rx');

var clickEvents = Rx.Observable
  .fromEvent(document.querySelector('#btn'), 'click')

var numberEvents = Rx.Observable
  .fromArray(numbers);

function pickSecond(c, n) { return n; }

Rx.Observable.zip(clickEvents, numberEvents, 
                  pickSecond)
  .map(byConstant)
  .subscribe(print);
// prints a number on each button click

Marble diagram for 

Rx.zip

zip(s1, s2, fn)
function searchWikipedia (term) {
    return $.ajax({
        url: 'http://en.wikipedia.org/w/api.php',
        dataType: 'jsonp',
        data: {
            action: 'opensearch',
            format: 'json',
            search: term
        }
    }).promise();
}

Autocomplete

Hard problem: need to throttle, handle out of order returned results, etc.

function searchWikipedia (term) {
    return $.ajax({
        url: 'http://en.wikipedia.org/w/api.php',
        dataType: 'jsonp',
        data: {
            action: 'opensearch',
            format: 'json',
            search: term
        }
    }).promise();
}
var keyups = Rx.Observable.fromEvent($('#input'), 'keyup')
    .map(function (e) { return e.target.value; })
    .filter(function (text) { return text.length > 2; });

Autocomplete

keyups.throttle(500)
    .distinctUntilChanged()
    .flatMapLatest(searchWikipedia)
    .subscribe(function (data) {
        // display results
    });

Rx is the machine gun,

events are the ammo

Summary

Write small pure functions

From procedural to RFP

Summary

​Handle single async events using promises

From procedural to RFP

Summary

​Handle multiple async event streams using Rx

From procedural to RFP

Say "Goodbye" to old friends

Tell return values Bye-bye

// imperative code
var sum = add(2, 3);
// can use sum right away!

var multiplied = [1, 2, 3].map(mul);
// can use multiplied array right away!
var _ = require('lodash');
var byConstant = _.partial(mul, constant);
_(numbers)
  .map(byConstant)
  .forEach(print);
// we have not returned any values!

glebbahmutov.com/blog/abandoning-return-values/

Tell middle functions Bye-bye

var byConstant = _.partial(mul, constant);
_(numbers)
  .map(function (x) {
    return byConstant(x);
  })
  .forEach(function (x) {
    print(x);
  });
_(numbers)
  .map(byConstant)
  .forEach(print);
// no middle functions

glebbahmutov.com/blog/point-free-is-not-point-less/

Tell deep equality checks Bye-bye*

var result = process(numbers);
if (deepEqual(result, numbers)) {
  render();
}
var result = process(immutableNumbers);
if (result === immutableNumbers) {
  render();
}

glebbahmutov.com/blog/avoid-side-effects-with-immutable-data-structures/

* with immutable data

Q: Which step causes the most difficulties?

A: For me it was abandoning the return values, others had problem with promises

glebbahmutov.com/blog/abandoning-return-values/

Q: How to nudge developers along the journey?

A: Having everyone interested train together

glebbahmutov.com/blog/overcoming-functional-programming-objections/

Future is bright

All examples shown use today's ES5 standard

ES6+ makes some pieces simpler

  • arrow notation
  • block-scoping and constants
  • spread operator
  • native Promises (but not async / await)
  • binding context in callbacks

Thank you 👏

Journey from procedural to reactive JavaScript with stops - CoderCruise 2019

By Gleb Bahmutov

Journey from procedural to reactive JavaScript with stops - CoderCruise 2019

Understand the differences between imperative, object-oriented, functional, reactive and other styles of programming JavaScript can mimic. Each style brings more power and elegance to your code, and I will show how to adopt the new paradigm without losing your mind in complexity.

  • 2,694

More from Gleb Bahmutov