Or how I learned to stop worrying and love JS
@liamzebedee | liamz.co
Distributing apps on the web is like sending the bible for every single church reading
Bundles > 2MB
Recent innovations:
Webpack
Yarn / NPM united packages
React
Styled-components
PWA’s
The future
Edge computing
Bundle identifiers (2abd79e7d9fg.js) and c-hash-ing
SplitChunks
Static analysis of dependencies into a graph
Split out into many <script src=> and load async
Minimization (FooBar -> a.b)
Prefetch/preload
HTTP2 Push
Problems:
Small changes => more download => less battery life (mobile)
Download a whole lot more than I need to
We don’t really cache code on webpages, so reloads are always slow
Oppportunity:
XKCD (web is instant apps) - AR etc.
WebAssembly structured stack machine
P2P CDN’s?
Deliver code for execution
What if we downloaded only what changed?
How?
Naive approach: delta
Knowledge of \( C_t \) for \( 0..t \)
Problem: How can we synchronise state without high bandwidth usage?
We already compare the hashes of assets, but this is for the entire file. Can we do it on a more granular scale
Merkle tree (popularised in blockchain world)
A tree data structure
Where every non-leaf node is hash of its children
Compilers turn code into lower-level code. Example:
// Compile JSX (React syntax) to JS (vanilla JavaScript)
//
source = fs.readFile('something.jsx')
// export default const blah = () => {
// return <div>Hello world</div>;
// }
compiled = runLoaders(src)
// module.exports = function blah() {
// return React.createElement('div', { children: "Hello world" })
// }
Compilation:
What does that look like?
function shout() {
console.log("BLOCKCHAIN!");
}
// AST
Function [name=shout]
Call [name=console.log]
Literal ["BLOCKCHAIN!"]
// Merkle AST
42
123
543
function shout() {
console.log("BLOCKCHAIN!");
}
// AST
Function [name=shout]
Call [name=console.log]
Literal ["BLOCKCHAIN!"]
// Merkle AST
42
123
543
function shout() {
console.log("BLOCKCHAIN!", "IOT!");
}
// AST
Function [name=shout]
Call [name=console.log]
Literal ["BLOCKCHAIN!"]
Literal ["IOT!"]
// Merkle AST
86
99
543
264
*changed
Send client tree to server, compare the Merkle AST's
86*
99*
543
264*
Client
42
123
543
Server
86
99
543
264
What's changed?
Send client to server, compare the Merkle AST's
86*
99*
543
264*
Client
42
123
543
Server
86
99
543
264
What's changed?
We send only the diffs to the client
We send only the diffs to the client
They patch their source
And run the bundle
Loading the bundle (client)
// GET /
tree = localStorage.get('tree') || {};
fetch('/bundle', { tree })
.then(res => {
tree.merge(res.changes)
webpackBootstrap(tree.modules())
})
class tree {
modules: { [id]: string } = {};
tree = {};
merge(changes) {
this.modules = changes.map((id, diffs) => {
let module = this.modules[id];
diffs.map({ from, to, content } => {
module = module.updateRange(from, to, content);
})
return module;
})
this.recomputeTree()
}
recomputeTree() {
for({ id, module } in Object.entries(this.modules)) {
this.trees[id] = buildMerkleAst(module);
})
}
get modules() {
return Object.values(this.modules);
}
}
Serving the bundle (server)
handle('/bundle', (req, res) => {
let trees = req.trees;
let changeset = trees.map((id, tree) => {
let module = modules[id];
let diff = treediff(module.tree, tree);
let changes = diff.nodes.map(node => {
let { from, to } = module.tree.getRange(node)
let diff = module.getCode(from, to);
return { from, to, diff }
})
return { id, changes }
})
res.send(changeset);
})
liamzebedee/go-json-merkle-tree - builds merkle tree of JSON blob (AST) and outputs differing nodes
webpack-merkle-ast (WIP)
hooks into Webpack build pipeline
merkle-ast-server (tbc)
serves these diff's to clients
Liam Zebedee
liamz.co