Caching for Cash 🤑

Kent C. Dodds

slides.com/kentcdodds/caching-for-cash

EpicReact.dev

More than you knew you need to know about caching

EpicWeb.dev

kentcdodds.com

Let's wake up

@kentcdodds

Your brain needs this 🧠

What this talk is

Deep dive on caching fundamentals
Code examples

@kentcdodds

What this talk is not

Comprehensive

@kentcdodds

Let's
Get
STARTED!

@kentcdodds

Two ways to make your code faster:

Delete it
Reduce the amount of stuff the code is doing

Delete it

Reduce the...

stuff

Can't delete it. Can't reduce it. Can't "make it fast."

🤑 Cash it! 🤑

🥴 Cache it! 🥴

What is caching?

In computing, a cache is a hardware or software component that stores data so that future requests for that data can be served faster; the data stored in a cache might be the result of an earlier computation or a copy of data stored elsewhere. A cache hit occurs when the requested data can be found in a cache, while a cache miss occurs when it cannot. Cache hits are served by reading data from the cache, which is faster than recomputing a result or reading from a slower data store; thus, the more requests that can be served from the cache, the faster the system performs.

Caching by example

function computePi() {
	let pi = 0
	let sign = 1
	for (let i = 0; i < 1000000; i++) {
		const term = sign / (2 * i + 1)
		pi += term
		sign *= -1
	}
	return Math.round(pi * 4e10) / 1e10
}

Caching by example

let pi
function computePiCached() {
	if (typeof pi === 'undefined') {
		pi = computePi()
	}
	return pi
}

Store the result of a computation somewhere and return that stored value instead of recomputing it again.

💡 Caching:

More complexity

function computePi(precision: number) {
	let pi = 0
	let sign = 1
	for (let i = 0; i < 1000000; i++) {
		pi += sign / (2 * i + 1)
		sign *= -1
	}
	const factor = 10 ** precision
	return Math.round(pi * 4 * factor) / factor
}

More complexity

const piCache = new Map<number, number>()
function computePiCached(precision: number) {
	if (!piCache.has(precision)) {
		piCache.set(precision, computePi(precision))
	}
	return piCache.get(precision)
}

🔑 Cache Keys!!

Another example

function sum(a: number, b: number) {
	 return a + b
}

const sumCache = new Map<string, number>()
function sumCached(a: number, b: number) {
	const key = `${a},${b}`
	if (!sumCache.has(key)) {
		sumCache.set(key, sum(a, b))
	}
	return sumCache.get(key)
}

sumCached(1, 2) // cache miss: 3
sumCached(1, 2) // cache hit: 3

The problem with keys

function addDays(count: number) {
  const millisecondsInDay = 1000 * 60 * 60 * 24
  return new Date(Date.now() + count * millisecondsInDay)
}

const cache = new Map<string, Date>()
function addDaysCached(count: number) {
  const key = `add-days:${count}`
  if (!cache.has(key)) {
    cache.set(key, addDays(count))
  }
  return cache.get(key)
}

find the bug 🐛

🐛

addDaysCached(3) // cache miss: 3 days from today
addDaysCached(3) // cache hit: 3 days from today
// ... wait 24 hours...
addDaysCached(3) // cache hit: 3 days from yesterday 😱
// That's 2 days from today!

The cache key must* account for all inputs required to determine the result

💡 Cache Keys:

*But...

It’s easy to miss an input
Too many inputs
Computing correct cache keys is costly

1. It’s easy to miss an input

useMemo(() => {
	// ...
}, [/* ... ugh... */])

2. Too many inputs

3. Computing correct cache keys is costly

So we cheat:

Cache revalidation

Cache Revalidation

Proactively Updating the Cache
On post update, update the cache
Timed Invalidation
Cache-Control headers
Stale While Revalidate
Update cache in the background
Forcing fresh value
Manual cache updates
Soft Purge 🤩
Manual Stale While Revalidate

Another caching problem

import fs from 'fs'

function getVideoBuffer(filepath: string) {
	return fs.promises.readFile(filepath)
}

const videoBufferCache = new Map<string, Buffer>()
async function getVideoBufferCached(filepath: string) {
	if (!videoBufferCache.has(filepath)) {
		videoBufferCache.set(filepath, await getVideoBuffer(filepath))
	}
	return videoBufferCache.get(filepath)
}

find the bug 🐛

🐛

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
 1: 00007FF6E7A84E05 node::Abort+21
 2: 00007FF6E7A84F49 node::OnFatalError+297
 3: 00007FF6E7A84D2F node::OnFatalError+31
 4: 00007FF6E7A84D17 node::OnFatalError+23
 5: 00007FF6E812C7F8 v8::Utils::ReportOOMFailure+184
...