Caching for Cash πŸ€‘

Kent C. Dodds

More than you knew you need to know about caching

Let's wake up

Your brain needs this 🧠

What this talk is

  • Deep dive on caching fundamentals
  • Code examples

What this talk is not

  • Comprehensive


Two ways to make your code faster:

  1. Delete it
  2. Reduce the amount of stuff the code is doing

Delete it

Reduce the...


Can't delete it. Can't reduce it. Can't "make it fast."

πŸ€‘ Cash it! πŸ€‘

πŸ₯΄ Cache it! πŸ₯΄

What is caching?

In computing, a cache is a hardware or software component that stores data so that future requests for that data can be served faster; the data stored in a cache might be the result of an earlier computation or a copy of data stored elsewhere. A cache hit occurs when the requested data can be found in a cache, while a cache miss occurs when it cannot. Cache hits are served by reading data from the cache, which is faster than recomputing a result or reading from a slower data store; thus, the more requests that can be served from the cache, the faster the system performs.

Caching by example

function computePi() {
	let pi = 0
	let sign = 1
	for (let i = 0; i < 1000000; i++) {
		const term = sign / (2 * i + 1)
		pi += term
		sign *= -1
	return Math.round(pi * 4e10) / 1e10

Caching by example

let pi
function computePiCached() {
	if (typeof pi === 'undefined') {
		pi = computePi()
	return pi

Store the result of a computation somewhere and return that stored value instead of recomputing it again.

πŸ’‘ Caching:

More complexity

function computePi(precision: number) {
	let pi = 0
	let sign = 1
	for (let i = 0; i < 1000000; i++) {
		pi += sign / (2 * i + 1)
		sign *= -1
	const factor = 10 ** precision
	return Math.round(pi * 4 * factor) / factor

More complexity

const piCache = new Map<number, number>()
function computePiCached(precision: number) {
	if (!piCache.has(precision)) {
		piCache.set(precision, computePi(precision))
	return piCache.get(precision)

πŸ”‘ Cache Keys!!

Another example

function sum(a: number, b: number) {
	 return a + b

const sumCache = new Map<string, number>()
function sumCached(a: number, b: number) {
	const key = `${a},${b}`
	if (!sumCache.has(key)) {
		sumCache.set(key, sum(a, b))
	return sumCache.get(key)

sumCached(1, 2) // cache miss: 3
sumCached(1, 2) // cache hit: 3

The problem with keys

function addDays(count: number) {
  const millisecondsInDay = 1000 * 60 * 60 * 24
  return new Date( + count * millisecondsInDay)

const cache = new Map<string, Date>()
function addDaysCached(count: number) {
  const key = `add-days:${count}`
  if (!cache.has(key)) {
    cache.set(key, addDays(count))
  return cache.get(key)

find the bug πŸ›


addDaysCached(3) // cache miss: 3 days from today
addDaysCached(3) // cache hit: 3 days from today
// ... wait 24 hours...
addDaysCached(3) // cache hit: 3 days from yesterday 😱
// That's 2 days from today!

The cache key must* account for all inputs required to determine the result

πŸ’‘ Cache Keys:


  1. It’s easy to miss an input
  2. Too many inputs
  3. Computing correct cache keys is costly

1. It’s easy to miss an input

useMemo(() => {
	// ...
}, [/* ... ugh... */])

2. Too many inputs

3. Computing correct cache keys is costly

So we cheat:

Cache revalidation

Cache Revalidation

  1. Proactively Updating the Cache
    On post update, update the cache
  2. Timed Invalidation
    Cache-Control headers
  3. Stale While Revalidate
    Update cache in the background
  4. Forcing fresh value
    Manual cache updates
  5. Soft Purge 🀩
    Manual Stale While Revalidate

Another caching problem

import fs from 'fs'

function getVideoBuffer(filepath: string) {
	return fs.promises.readFile(filepath)

const videoBufferCache = new Map<string, Buffer>()
async function getVideoBufferCached(filepath: string) {
	if (!videoBufferCache.has(filepath)) {
		videoBufferCache.set(filepath, await getVideoBuffer(filepath))
	return videoBufferCache.get(filepath)

find the bug πŸ›


FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
 1: 00007FF6E7A84E05 node::Abort+21
 2: 00007FF6E7A84F49 node::OnFatalError+297
 3: 00007FF6E7A84D2F node::OnFatalError+31
 4: 00007FF6E7A84D17 node::OnFatalError+23
 5: 00007FF6E812C7F8 v8::Utils::ReportOOMFailure+184

Cache Size Solutions

  1. Least Recently Used
    Ditch old stuff (
  2. File System
    require('os').tmpdir() or ./node_modules/.cache
  3. SQLite
    Even distributed with LiteFS
  4. Redis
    Super common, very fast, "more than a cache"

Note: cache size can still get out of control, so keep an eye out!

Cache Warming


  1. You can get rate limited by APIs
  2. It requires a lot of resources
  3. Making users wait for the fresh values

Solution: Soft Purge

Cache Entry Value Validation

Cache Request Deduplication

Kinda like DataLoader

You're looking for cachified

Thank you!