Caching for Cash 🤑
Kent C. Dodds
More than you knew you need to know about caching
Let's wake up
Your brain needs this 🧠
What this talk is
- Deep dive on caching fundamentals
- Code examples
What this talk is not
- Comprehensive
Let's
Get
STARTED!
Two ways to make your code faster:
- Delete it
- Reduce the amount of stuff the code is doing
Delete it
Reduce the...
stuff
Can't delete it. Can't reduce it. Can't "make it fast."
🤑 Cash it! 🤑
🥴 Cache it! 🥴
What is caching?
In computing, a cache is a hardware or software component that stores data so that future requests for that data can be served faster; the data stored in a cache might be the result of an earlier computation or a copy of data stored elsewhere. A cache hit occurs when the requested data can be found in a cache, while a cache miss occurs when it cannot. Cache hits are served by reading data from the cache, which is faster than recomputing a result or reading from a slower data store; thus, the more requests that can be served from the cache, the faster the system performs.
Caching by example
function computePi() {
let pi = 0
let sign = 1
for (let i = 0; i < 1000000; i++) {
const term = sign / (2 * i + 1)
pi += term
sign *= -1
}
return Math.round(pi * 4e10) / 1e10
}Caching by example
let pi
function computePiCached() {
if (typeof pi === 'undefined') {
pi = computePi()
}
return pi
}Store the result of a computation somewhere and return that stored value instead of recomputing it again.
💡 Caching:
More complexity
function computePi(precision: number) {
let pi = 0
let sign = 1
for (let i = 0; i < 1000000; i++) {
pi += sign / (2 * i + 1)
sign *= -1
}
const factor = 10 ** precision
return Math.round(pi * 4 * factor) / factor
}More complexity
const piCache = new Map<number, number>()
function computePiCached(precision: number) {
if (!piCache.has(precision)) {
piCache.set(precision, computePi(precision))
}
return piCache.get(precision)
}🔑 Cache Keys!!
Another example
function sum(a: number, b: number) {
return a + b
}
const sumCache = new Map<string, number>()
function sumCached(a: number, b: number) {
const key = `${a},${b}`
if (!sumCache.has(key)) {
sumCache.set(key, sum(a, b))
}
return sumCache.get(key)
}
sumCached(1, 2) // cache miss: 3
sumCached(1, 2) // cache hit: 3The problem with keys
function addDays(count: number) {
const millisecondsInDay = 1000 * 60 * 60 * 24
return new Date(Date.now() + count * millisecondsInDay)
}
const cache = new Map<string, Date>()
function addDaysCached(count: number) {
const key = `add-days:${count}`
if (!cache.has(key)) {
cache.set(key, addDays(count))
}
return cache.get(key)
}find the bug 🐛
🐛
addDaysCached(3) // cache miss: 3 days from today
addDaysCached(3) // cache hit: 3 days from today
// ... wait 24 hours...
addDaysCached(3) // cache hit: 3 days from yesterday 😱
// That's 2 days from today!The cache key must* account for all inputs required to determine the result
💡 Cache Keys:
*But...
- It’s easy to miss an input
- Too many inputs
- Computing correct cache keys is costly
1. It’s easy to miss an input
useMemo(() => {
// ...
}, [/* ... ugh... */])2. Too many inputs
3. Computing correct cache keys is costly
So we cheat:
Cache revalidation
Cache Revalidation
- Proactively Updating the Cache
On post update, update the cache - Timed Invalidation
Cache-Control headers - Stale While Revalidate
Update cache in the background - Forcing fresh value
Manual cache updates - Soft Purge 🤩
Manual Stale While Revalidate
Another caching problem
import fs from 'fs'
function getVideoBuffer(filepath: string) {
return fs.promises.readFile(filepath)
}
const videoBufferCache = new Map<string, Buffer>()
async function getVideoBufferCached(filepath: string) {
if (!videoBufferCache.has(filepath)) {
videoBufferCache.set(filepath, await getVideoBuffer(filepath))
}
return videoBufferCache.get(filepath)
}find the bug 🐛
🐛
FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
1: 00007FF6E7A84E05 node::Abort+21
2: 00007FF6E7A84F49 node::OnFatalError+297
3: 00007FF6E7A84D2F node::OnFatalError+31
4: 00007FF6E7A84D17 node::OnFatalError+23
5: 00007FF6E812C7F8 v8::Utils::ReportOOMFailure+184
...Cache Size Solutions
- Least Recently Used
Ditch old stuff (npm.im/lru-cache) - File System
require('os').tmpdir() or ./node_modules/.cache -
SQLite
Even distributed with LiteFS -
Redis
Super common, very fast, "more than a cache"
Note: cache size can still get out of control, so keep an eye out!
Cache Warming
Problems
- You can get rate limited by APIs
- It requires a lot of resources
- Making users wait for the fresh values
Solution: Soft Purge
Cache Entry Value Validation
Cache Request Deduplication
Kinda like DataLoader
You're looking for cachified
Thank you!
Caching for Cash 🤑
By Kent C. Dodds
Caching for Cash 🤑
It's often said that the two hardest problems in programming are caching, naming things, and off by one errors. Some degree of caching is required in almost every application to drastically improve performance. Unfortunately, not only is it easy to get wrong, there are also lots of different layers and methods to implement caching with different trade-offs. In this talk, Kent will explain the key principles of caching. We'll go through several real world examples of issues where caching is a great solution. We'll also explore different approaches and their associated trade-offs. We'll cover a ton of ground here. This one's gonna be fun!
