LearnedCache:
ML-driven Page Cache Eviction Policies with eBPF
Jayden (Zejia) Qi
Eviction Policies

- Decide what cache entries to evict when full
- LRU, FIFO
- Tracked with circular doubly linked lists

Eviction Policies
... many of them




There is no "one-size-fits-all"
- A "perfect" algorithm evicts the page that will be reused the furthest in the future
- LRU/FIFO trails Belady's optimal algorithm by large margin
There is no "one-size-fits-all"
But maybe it's possible


- HALP (among other works) introduces policies for userspace caches
- Finds heuristics + ML enhancements to balance overhead and decision quality
eBPF (Extended Berkeley Package Filter)
- Allows for dynamically loaded, user-created C programs to run in Kernel space via the eBPF runtime
- Compiled to eBPF bytecode, then verified by the verifier
- pointer safety
- bounded loops
- stack size + program size limits


Provides a way for user-created eBPF programs to run as eviction policies.
SEC(".struct_ops.link")
struct cache_ext_ops fifo_ops = {
.init = (void *)fifo_init,
.evict_folios = (void *)fifo_evict_folios,
.folio_evicted = (void *)fifo_folio_evicted,
.folio_added = (void *)fifo_folio_added,
};void BPF_STRUCT_OPS(fifo_evict_folios, struct cache_ext_eviction_ctx *eviction_ctx,
struct mem_cgroup *memcg)
{
u64 lat_start = bpf_ktime_get_ns();
if (bpf_cache_ext_list_iterate(memcg, main_list, bpf_fifo_evict_cb, eviction_ctx) < 0)
bpf_printk("cache_ext: evict: Failed to iterate main_list\n");
}LearnedCache
- Accurate and performant neural models can be trained using kernel trace data of varying workloads.
- eBPF-driven implementation of policies utilizing such models is possible.
- Still show significant improvement over heuristic policies.
Framing the decision
Given a page at eviction time, will it be accessed within a set amount of time later?
- Can build training data offline using only a stream of accesses/evictions
- Can track features per-page online during benchmarking/inference
Tracers
Tracers (bolted onto FIFO policy) collect:
- Delta time between last accesses of page.
- Delta time between last 2 accesses of page (i.e. second and third to last).
- Delta time between last access of inode.
- Delta time between last 2 accesses of inode (i.e. second
- and third to last).
- Distance from last access within file.
- File size in pages.
- Relative exponential moving average for page accesses.
- Relative exponential moving average for inode accesses.
- Time from last access to eviction.

Model: Simple perceptron
p = \sigma(\mathbf{x} \cdot \mathbf{w} + b)

Model: Simple perceptron
p = \sigma(\mathbf{x} \cdot \mathbf{w} + b)
- Logistic regression (probability of reuse) during training time
- Sigmoid too expensive to call in kernel space (Needs taylor approximation/lookup table), so we operate on raw logits instead
Policies: Rank or Protect


Evaluation
- Evaluation done with LevelDB
- For each distinct workload
- Trace collected
- Model trained
- Reran with model policies
Evaluation - YSCB

- Yahoo! Cloud Serving Benchmark
- Follows Zipfian distribution (few pages get accessed most)
- A, F write-heavy
Evaluation - Latency
- Ranking sees large overhead, protect not as much
- Overhead is noticeable with larger cache pressure

Evaluation - Twitter Traces
- Twitter production Cache traces
- Lot more cache pressure -> ~6-8x ins/sec
- No heuristic performs best

Reflection, Limitations, Future Work
- Overfitting - needs train/test separation
- Hard to fit to changing workloads
- Demonstrates that neural prediction can be run relatively easily in kernel space
- Is it possible to create for other kernel components?
- Task scheduler, prefetch
Thank You!
github.com/JayAndJef
deck
By Jayden Q
deck
- 45