Gleb Bahmutov PRO
JavaScript ninja, image processing expert, software quality fanatic
A Journey From Procedural To Reactive JavaScript With (many) Stops
Gleb Bahmutov
Global temperature anomaly 2022 https://climate.nasa.gov/vital-signs/global-temperature/
survival is possible* but we need to act now
- change your life
- join an organization
- VOTE
Coding in JS
source: Improved Minesweeper
JavaScript == table
JavaScript < table
We use JS because it is everywhere
-
browser
-
server
-
mobile
-
robots
source: careers.stackoverflow.com
JavaScript / NPM growth
source: modulecounts.com for more: https://blog.sandworm.dev/state-of-npm-2023-the-overview
Changing how JS developers program will have a huge impact on the software quality in the world
Problem: code complexity is quickly growing
number of engineers
lifetime of the company
We need JavaScript to
- be as clear as writing an English sentence (or clearer)
- produce the expected result
- be simple to test
- assemble complex logic from very simple parts
Let's use FP to implement Reactive Programming patterns, like Netflix does!
- ME (too cheerfully)
FP in JS
- small pure functions
- compose complex logic from simple steps
- semi-performant async with promises
- simple(r) data with immutable structures
reactive programming
- every user action is stream of events
- every server action is stream of events
- streams can be processed using standard operations
2 major parts
Dev team (otherwise smart people)
Everyone has their own path to JS
Everyone is different
- 100% jQuery
- 30% FP in JS
- 25% OO in JS
! junior vs senior
- 50% jQuery
- 90% CSS
- 5% OO in JS
- 20% jQuery
- 30% CSS
- 75% OO in JS
- 90% FP in JS
In theory
anyone
functional reactive programmer
In practice
anyone
functional reactive programmer
...
...
...
...
...
...
...
- procedural
- object-oriented
- functional
- abandon return values
- immutable data
- lazy evaluation
- lazy async values
- promises
- event emitters
- event emitters returning promises
- chained event emitters
- reactive streams
Here is my journey through the JS land
Show me the code!
Problem: given an array of numbers, multiply each number by a constant and print the result.
var numbers = [3, 1, 7];
var constant = 2;
// 6 2 14more than 20 implementations at
procedural / imperative JS
var numbers = [3, 1, 7];
var constant = 2;
var k = 0;
for(k = 0; k < numbers.length; k += 1) {
console.log(numbers[k] * constant);
}
// 6 2 14-
"for" loop is a pain to debug and use
-
mixing data manipulation with printing
-
code is hard to reuse
-
Start splitting code into procedures
var numbers = [3, 1, 7];
var constant = 2;
function mul(a, b) {
return a * b;
}
function print(x) {
console.log(x);
}
function processNumber(n) {
print(mul(n, constant));
}
for(k = 0; k < numbers.length; k += 1) {
processNumber(numbers[k]);
}
// 6 2 14Start splitting code into procedures
var numbers = [3, 1, 7];
var constant = 2;
function mul(a, b) {
return a * b;
}
function print(x) {
console.log(x);
}
function processNumber(n) {
print(mul(n, constant));
}
for(k = 0; k < numbers.length; k += 1) {
processNumber(numbers[k]);
}
// 6 2 14reusable simple functions
Start splitting code into procedures
var numbers = [3, 1, 7];
var constant = 2;
function mul(a, b) {
return a * b;
}
function print(x) {
console.log(x);
}
function processNumber(n) {
print(mul(n, constant));
}
for(k = 0; k < numbers.length; k += 1) {
processNumber(numbers[k]);
}
// 6 2 14reusable simple functions
complex logic
Start splitting code into procedures
var numbers = [3, 1, 7];
var constant = 2;
function mul(a, b) {
return a * b;
}
function print(x) {
console.log(x);
}
function processNumber(n) {
print(mul(n, constant));
}
for(k = 0; k < numbers.length; k += 1) {
processNumber(numbers[k]);
}
// 6 2 14reusable simple functions
complex logic
iteration
Our goal
make the code boring
Boring code does not surprise you
// using "this" WILL surprise you
numbers.forEach(function (x) {
this.doSomething(x);
// ERROR!
});var self = this;
numbers.forEach(function (x) {
self.doSomething(x);
});Boring code does not surprise you
// using "this" in your objects
// WILL surprise someone else
numbers.forEach(console.log);numbers.forEach(console.log.bind(console));// works under Node
// in Chrome
// Uncaught TypeError: Illegal invocationIt was finally fixed a few years ago 👍
Boring code does not surprise you
// for loops WILL surprise you
for(k = 0; k < numbers.length; k += 1) {
processNumber(numbers[k]);
}common problems: off by 1 errors, iterator variable, changing array size
Boring code does not surprise you
// using non-local variables WILL surprise you
constant = 2;
...
numbers.forEach(function process(n) {
return mul(n, constant);
});common problems: undefined values, or values that change suddenly
Pure functions are really boring
function mul(a, b) {
return a * b;
}
function print(n) {
console.log(n);
}- output depends ONLY on the inputs
- same result every time
- no surprises
- pure against a set of unit tests
not really pure, but close enough
Boring functions make new boring functions using fn.bind
var mulBy2 = mul.bind(null, 2);
mulBy2(5); // 10function mul(a, b) {
return a * b;
}
var mul2by3 = mul.bind(null, 2, 3);
mul2by3(); // 6var mul2By10 = mulBy2.bind(null, 10);
mul2By10('extra', 'args are', 'ignored'); // 20Partial application
function mul(a, b) {
return a * b;
}
var by2 = mul.bind(null, 2);
// same as
var _ = require('lodash');
var by2 = _.partial(mul, 2);Boring code is nice, but I want encapsulated logic, data access, inheritance!
- a jaded developer
function NumberMultiplier() {}
NumberMultiplier.prototype.setNumbers = function (numbers) {
this.numbers = numbers;
return this;
};
NumberMultiplier.prototype.multiply = function (constant) {
for (var k = 0; k < this.numbers.length; k += 1) {
this.numbers[k] = constant * this.numbers[k];
}
return this;
};
NumberMultiplier.prototype.print = function () {
console.log(this.numbers);
};Object-Oriented JavaScript
Object-Oriented JavaScript
// using NEW keyword
new NumberMultiplier()
.setNumbers(numbers)
.multiply(constant)
.print();
// [ 6, 2, 14 ]Object-Oriented JavaScript: 3 parts
function NumberMultiplier() {}
NumberMultiplier.prototype.setNumbers = function() {}
NumberMultiplier.prototype.multiply = function() {}
NumberMultiplier.prototype.print = function() {}
new NumberMultiplier();constructor function
prototype
"new" keyword
// NumberMultiplier.prototype object
{
setNumbers: function () { ... },
multiply: function () { ... },
print: function () { ... }
}
/\
|| [[ prototype ]]
||
// instance object
{
numbers: [3, 1, 7]
}OO JavaScript is prototypical
var NumberMultiplier = {
setNumbers: function (numbers) {
this.numbers = numbers; return this;
},
multiply: function (constant) {
for (var k = 0; k < this.numbers.length; k += 1) {
this.numbers[k] = constant * this.numbers[k];
} return this;
},
print: function () { console.log(this.numbers); }
};Prototype is a plain object!
var numberMultiplier = Object.create(NumberMultiplier);
numberMultiplier
.setNumbers(numbers)
.multiply(constant)
.print();
// [ 6, 2, 14 ]plain object
class NumberMultiplier {
setNumbers(numbers) {
this.numbers = numbers;
return this;
}
multiply(constant) {
for (var k = 0; k < this.numbers.length; k += 1) {
this.numbers[k] = constant * this.numbers[k];
}
return this;
}
print() {
console.log(this.numbers);
return this;
}
}
new NumberMultiplier()ES6 classes: just don't
JavaScript Arrays
typeof [1, 2] // "object"Array.isArray([1, 2]) // trueArray.prototype
/*
map
forEach
filter
reduce
...
*/Arrays: object-oriented + functional
var numbers = [3, 1, 7];
var constant = 2;
function mul(a, b) {
return a * b;
}
function print(n) {
console.log(n);
}
var mulBy = mul.bind(null, constant);
numbers
.map(mulBy)
.forEach(print);
// 6 2 14- clear "multiply then print" semantics
Arrays: object-oriented + functional
var numbers = [3, 1, 7];
var constant = 2;
function mul(a, b) {
return a * b;
}
function print(n) {
console.log(n);
}
var mulBy = mul.bind(null, constant);
numbers
.map(mulBy)
.forEach(print);
// 6 2 14- clear "multiply then print" semantics
functional bits
Arrays: object-oriented + functional
var numbers = [3, 1, 7];
var constant = 2;
function mul(a, b) {
return a * b;
}
function print(n) {
console.log(n);
}
var mulBy = mul.bind(null, constant);
numbers
.map(mulBy)
.forEach(print);
// 6 2 14- clear "multiply then print" semantics
functional bits
Object-oriented methods
My advice
-
Write small pure functions
-
Adapt them to particular call sites
-
Progress slowly through the code base
Observation: a lot of our code
manipulates collections of data items
// OO (built-in)
Array.prototype.map(cb)
// functional (user space)
Library.map(array, cb)
// plus
Library.groupBy(array, cb);
Library.dropWhile(array, cb);
...
2 of the top 3 most dependent upon NPM projects are FP utility libraries: underscore, lodash
var _ = require('lodash');
var byConstant = _.partial(mul, constant);
// _.forEach(array, cb);
// _.map(array, cb);Work with arrays using Lodash
_.forEach(
_.map(numbers, byConstant),
print
);
// 6 2 14_({ John: 3, Jim: 10 })
.map(byConstant)
.forEach(print);
// 6 20Treat objects same as collections using lodash
Argument order
var mulBy = ...
// OO (built-in)
Array.prototype.map(mulBy)
// lodash / underscore
_.map(array, mulBy)
// Ramda
R.map(mulBy, array)var R = require('ramda');
var byConstant = R.partial(R.mul, constant);1. compose algorithm
2. apply to data
3. profit!!!
// R.forEach = function (callback, array)
var printEach = R.partial(R.forEach, print);// R.map: function (callback, array)
var mapByConstant = R.partial(R.map, byConstant);composition
printEach(mapByConstant(numbers));
// 6 2 14var R = require('ramda');
var byConstant = R.partial(R.mul, constant);1. compose algorithm
2. apply to data
3. profit!!!
// R.forEach = function (callback, array)
var printEach = R.partial(R.forEach, print);
var algorithm = R.compose(printEach, mapByConstant);
algorithm(numbers);
// 6 2 14// R.map: function (callback, array)
var mapByConstant = R.partial(R.map, byConstant);// fn( data )
var R = require('ramda');
var byConstant = R.partial(R.mul, constant);// R.forEach = function (callback, array)
var printEach = R.partial(R.forEach, print);
var algorithm = R.compose(printEach, mapByConstant);
algorithm(numbers);
// 6 2 14// R.map: function (callback, array)
var mapByConstant = R.partial(R.map, byConstant);"fn(data)" code is simple to reason about and test
// fn( data )
// R.mul = function (a, b) { ... }
var byConstant = R.partial(R.mul, constant);
Partial application vs curry
// in Ramda this is same as simply
var byConstant = R.mul(constant);
// for your functions
var mul = R.curry(function (a, b) {
return a * b;
});
mul(5)(4); // 20
var R = require('ramda');
R.pipe(
R.map(R.multiply(constant)),
R.forEach(print)
)(numbers);
// 6 2 14Curry: program with very little code
Requires careful function signature design
i.e. put information most likely to be known early as first argument
OO
"Smart" objects
Pass data around
FP
"Dumb" objects
"Smart" pipeline
Pass code (functions) around
new NumberMultiplier()
.add(3, 1, 7)
.multiply(2)
.print();_.forEach(
_.map(numbers, byConstant),
print
);Object-oriented VS functional
It is better to have 100 functions operate on one data structure than to have 10 functions operate on 10 data structures."
- Alan J. Perlis (Algol)
new NumberMultiplier()
.add(3, 1, 7)
.multiply(2)
.print();_.forEach(
_.map(numbers, byConstant),
print
);What else can JavaScript do for us?
Source of complexity: mutable data
var numbers = [...];
numbers.forEach(processNumber);
// is the list "numbers" unchanged?function processNumber(x, k, array) {
array[1] = 10000;
print(x);
}Mutable data makes code hard to understand and modify
Immutable data structures
'use strict';
var immutable = require('seamless-immutable');
var byConstant = _.partial(mul, constant);
immutable(numbers)
.map(byConstant)
.forEach(function (x, k, array) {
array[1] = 10000; // throws an Error
print(x);
});Efficient immutable JS libraries: seamless-immutable, mori
Immutable data with Redux pattern
function mul(a, b) { return a * b; }
function multiply(state, constant) {
var byConstant = mul.bind(null, constant);
return state.updateIn(['numbers'], function (ns) {
return ns.map(byConstant);
});
}var immutable = require('immutable');
var initialState = immutable.Map({
numbers: immutable.List.of([3, 1, 7])
});
var newState = multiply(initialState, constant);
// newState !== initialStateArrays: great for synchronous actions
Efficient computation with transducers
Do as little as needed with lazy evaluation
Arrays: NOT SO great for asynchronous
or infinite data!
// Does NOT work
numbers.map(function (x, done) {
mul(x, function cb(result) {
done(result);
})
}).forEach(function (value) {
print(value, function cb() {
done():
})
});How to handle async callbacks during iteration?
numbers
.map(asyncFn)
.forEach(asyncFn)Promises
const p = new Promise((resolve, reject) => {
...
resolve(42)
})
// after promise successfully completes
p.then(...)
.catch(...)Single async operation with a value or an error
Each callback can be synchronous or return a Promise
function sleep () {
return new Promise(resolve => {
setTimeout(resolve, 1000)
})
}
p.then(sleep)var sleepSecond = _.partial(Q.delay, 1000);
var pauseMulAndPrint = function (n) {
return function () {
return sleepSecond()
.then(_.partial(byConstant, n))
.then(print);
};
};Sleep, mul then print using promises
numbers.map(pauseMulAndPrint)
.reduce(Q.when, Q())
.done();
// ... 6 ... 2 ... 14Promises handle single event very well
Extra complexity trying to handle sequence of events
var sleepSecond = _.partial(Q.delay, 1000);
var pauseMulAndPrint = function (n) {
return function () {
return sleepSecond()
.then(_.partial(byConstant, n))
.then(print);
};
};
numbers.map(pauseMulAndPrint)
.reduce(Q.when, Q())
.done();
// ... 6 ... 2 ... 14Constructing a promise for each number :(
var sleepSecond = _.partial(Q.delay, 1000);
var pauseMulAndPrint = function (n) {
return function () {
return sleepSecond()
.then(_.partial(byConstantAsync, n))
.then(print);
};
};
Promise is like a single cartridge
Improve upon promises: need to chain async actions without boilerplate per item
"shoot the same bullet more than once"
Event Emitters
-
handle repeated events
-
decouple the event source from an action
Widely used in browser code
$(el).on('click', action) window.on('resize', cb) socket.on('message', handle)
We can extend the same approach to any event generator (like number sequence)
// action
var events = require('events');
var numberEmitter = new events.EventEmitter();
numberEmitter.on('number',
_.compose(print, byConstant)
);Single code pipeline
var k = 0;
var ref = setInterval(function () {
numberEmitter.emit('number', numbers[k++]);
if (k >= numbers.length) {
clearInterval(ref);
}
}, 1000);
// prints 6, 2 14 with 1 second intervalsGoal: process number with 1 second pauses
Our emitter has `on`
- What is the semantic meaning of `on`?
- How do we connect multiple steps?
numberEmitter
.on('number', action);Remember Array ES5 methods?
[3, 1, 7]
.map(mulBy2)
.forEach(print);I want this!
source of number events
.map(an async callback)
.buffer(n, async callback)
.forEach(another async callback);Chained emitters
var stepEmitter = {
map: function (cb) {
var emitter = new events.EventEmitter();
return _.extend(emitter, stepEmitter);
},
forEach: function (cb) {
var emitter = new events.EventEmitter();
return _.extend(emitter, stepEmitter);
}
};Chained emitters
var stepEmitter = {
map: function (cb) {
var emitter = new events.EventEmitter();
this.on('step', function (value) {
var mappedValue = cb(value);
emitter.emit('step', mappedValue);
});
return _.extend(emitter, stepEmitter);
},
forEach: function (cb) {
var emitter = new events.EventEmitter();
this.on('step', function (value) {
cb(value);
emitter.emit('step', value);
});
return _.extend(emitter, stepEmitter);
}
};Chained emitters
var stepEmitter = {
map: function (cb) {
var emitter = new events.EventEmitter();
this.on('step', function (value) {
var mappedValue = cb(value);
emitter.emit('step', mappedValue);
});
return _.extend(emitter, stepEmitter);
},
forEach: function (cb) {
var emitter = new events.EventEmitter();
this.on('step', function (value) {
cb(value);
emitter.emit('step', value);
});
return _.extend(emitter, stepEmitter);
}
};.map
emits cb(x)
.forEach
emits x
source of number events
.map(syncFn)
.forEach(syncFn); Chained emitters
So far: steps are async, but each callback function is synchronous!
Async callback support via promises
var Q = require('q');
var stepEmitter = {
map: function (cb) {
var emitter = new events.EventEmitter();
this.on('step', function (value) {
Q.when(cb(value)).then(function (mappedValue) {
emitter.emit('step', mappedValue);
});
});
return _.extend(emitter, stepEmitter);
},
forEach: function (cb) {
var emitter = new events.EventEmitter();
this.on('step', function (value) {
Q.when(cb(value)).then(function () {
emitter.emit('step', value);
});
});
return _.extend(emitter, stepEmitter);
}
};source of number events
.map(asyncFn)
.forEach(asyncFn); Chained emitters + promises
async for the win!
var stepEmitter = {
// same map and forEach methods as above
// returns a new step emitter that accumulates N items
// emits the entire array with N items as single argument
buffer: function (n) {
var received = [];
var emitter = new events.EventEmitter();
this.on('step', function (value) {
received.push(value);
if (received.length === n) {
emitter.emit('step', received);
received = [];
}
});
return _.extend(emitter, stepEmitter);
}
};
source(numbers)
.map(byConstant)
.buffer(3)
.forEach(print);
// sleeps 3 seconds then prints [6, 2, 14]What about buffering events?
source(numbers)
.map(byConstant) // data transform
.buffer(3) // sequence control
.forEach(print); // data transformWe implemented 2 types of operations on a sequence of events
Need extra methods to control the sequence (stream) of events
.map, .filter
// vs
.bufferReactive programming
- Everything is a source of asynchronous events.
- Your code is a pipeline reacting to one or more input streams and outputting another stream of events
Reactive JS libraries
- collection processing methods (map, filter, ...)
- stream control methods (buffer, zip, flatMapLatest, ...)
Use a reactive FP library
var Rx = require('rx');
var timeEvents = Rx.Observable
.interval(1000)
.timeInterval(); // stream 1var numberEvents = Rx.Observable
.fromArray(numbers); // stream 2Rx.Observable.zip(timeEvents, numberEvents,
function pickValue(t, n) { return n; })
.map(byConstant)
.subscribe(print); // stream 3
// prints 6 2 14 with 1 second intervalsUser click events
var Rx = require('rx');
var clickEvents = Rx.Observable
.fromEvent(document.querySelector('#btn'), 'click')
var numberEvents = Rx.Observable
.fromArray(numbers);
function pickSecond(c, n) { return n; }
Rx.Observable.zip(clickEvents, numberEvents,
pickSecond)
.map(byConstant)
.subscribe(print);
// prints a number on each button click
Marble diagram for
zip(s1, s2, fn)
function searchWikipedia (term) {
return $.ajax({
url: 'http://en.wikipedia.org/w/api.php',
dataType: 'jsonp',
data: {
action: 'opensearch',
format: 'json',
search: term
}
}).promise();
}Autocomplete
Hard problem: need to throttle, handle out of order returned results, etc.
function searchWikipedia (term) {
return $.ajax({
url: 'http://en.wikipedia.org/w/api.php',
dataType: 'jsonp',
data: {
action: 'opensearch',
format: 'json',
search: term
}
}).promise();
}var keyups = Rx.Observable.fromEvent($('#input'), 'keyup')
.map(function (e) { return e.target.value; })
.filter(function (text) { return text.length > 2; });
Autocomplete
keyups.throttle(500)
.distinctUntilChanged()
.flatMapLatest(searchWikipedia)
.subscribe(function (data) {
// display results
});
Rx is the machine gun,
events are the ammo
Summary
Write small pure functions
From procedural to RFP
Summary
Handle single async events using promises
From procedural to RFP
Summary
Handle multiple async event streams using Rx
From procedural to RFP
Say "Goodbye" to old friends
- Return values
- Boilerplate functions
- Some deep equality checks
Tell return values Bye-bye
// imperative code
var sum = add(2, 3);
// can use sum right away!
var multiplied = [1, 2, 3].map(mul);
// can use multiplied array right away!var _ = require('lodash');
var byConstant = _.partial(mul, constant);
_(numbers)
.map(byConstant)
.forEach(print);
// we have not returned any values!Tell middle functions Bye-bye
var byConstant = _.partial(mul, constant);
_(numbers)
.map(function (x) {
return byConstant(x);
})
.forEach(function (x) {
print(x);
});_(numbers)
.map(byConstant)
.forEach(print);
// no middle functionsTell deep equality checks Bye-bye*
var result = process(numbers);
if (deepEqual(result, numbers)) {
render();
}var result = process(immutableNumbers);
if (result === immutableNumbers) {
render();
}* with immutable data
Q: Which step causes the most difficulties?
A: For me it was abandoning the return values, others had problem with promises
Q: How to nudge developers along the journey?
A: Having everyone interested train together
Future is bright
const getTodo = (
id: number
): Effect.Effect<
unknown,
HttpClientError | TimeoutException
> =>
Http.request.get(`/todos/${id}`).pipe(
Http.client.fetchOk,
Http.response.json,
Effect.timeout("1 second"),
Effect.retry(
Schedule.exponential(1000).pipe(
Schedule.compose(Schedule.recurs(3)),
),
),
) Future is bright
fetching data with retries and timeouts
👏 Thank you 👏
Gleb Bahmutov
Journey from procedural to reactive JavaScript with stops
By Gleb Bahmutov
Journey from procedural to reactive JavaScript with stops
JavaScript is an interesting language. It can mimic almost any style you want: procedural, object-oriented, functional, etc. In this presentation, I will take a simple problem and will solve it using different approaches. With each step, we will see the power of each approach to take the complexity away, while still being the JavaScript we all love to hate. Presented at iJS May 2024, 30 minutes
