LearnedCache:

ML-driven Page Cache Eviction Policies with eBPF

Jayden (Zejia) Qi

Eviction Policies

  • Decide what cache entries to evict when full
  • LRU, FIFO
  • Tracked with circular doubly linked lists

Eviction Policies

... many of them

There is no "one-size-fits-all"

  • A "perfect" algorithm evicts the page that will be reused the furthest in the future
  • LRU/FIFO trails Belady's optimal algorithm by large margin

There is no "one-size-fits-all"

But maybe it's possible

  • HALP (among other works) introduces policies for userspace caches
  • Finds heuristics + ML enhancements to balance overhead and decision quality

eBPF (Extended Berkeley Package Filter)

  • Allows for dynamically loaded, user-created C programs to run in Kernel space via the eBPF runtime
  • Compiled to eBPF bytecode, then verified by the verifier
    • pointer safety
    • bounded loops
    • stack size + program size limits

Provides a way for user-created eBPF programs to run as eviction policies. 

SEC(".struct_ops.link")
struct cache_ext_ops fifo_ops = {
	.init = (void *)fifo_init,
	.evict_folios = (void *)fifo_evict_folios,
	.folio_evicted = (void *)fifo_folio_evicted,
	.folio_added = (void *)fifo_folio_added,
};
void BPF_STRUCT_OPS(fifo_evict_folios, struct cache_ext_eviction_ctx *eviction_ctx,
		    struct mem_cgroup *memcg)
{
	u64 lat_start = bpf_ktime_get_ns();
	if (bpf_cache_ext_list_iterate(memcg, main_list, bpf_fifo_evict_cb, eviction_ctx) < 0)
		bpf_printk("cache_ext: evict: Failed to iterate main_list\n");
}

LearnedCache

  1. Accurate and performant neural models can be trained using kernel trace data of varying workloads.
  2. eBPF-driven implementation of policies utilizing such models is possible.
  3. Still show significant improvement over heuristic policies.

Framing the decision

Given a page at eviction time, will it be accessed within a set amount of time later?

  • Can build training data offline using only a stream of accesses/evictions
  • Can track features per-page online during benchmarking/inference

Tracers

Tracers (bolted onto FIFO policy) collect:

  • Delta time between last accesses of page.
  • Delta time between last 2 accesses of page (i.e. second and third to last).
  • Delta time between last access of inode.
  • Delta time between last 2 accesses of inode (i.e. second
  • and third to last).
  • Distance from last access within file.
  • File size in pages.
  • Relative exponential moving average for page accesses.
  • Relative exponential moving average for inode accesses.
  • Time from last access to eviction.

Model: Simple perceptron

p = \sigma(\mathbf{x} \cdot \mathbf{w} + b)

Model: Simple perceptron

p = \sigma(\mathbf{x} \cdot \mathbf{w} + b)
  • Logistic regression (probability of reuse) during training time
  • Sigmoid too expensive to call in kernel space (Needs taylor approximation/lookup table), so we operate on raw logits instead

Policies: Rank or Protect

Evaluation

  • Evaluation done with LevelDB
  • For each distinct workload
    • Trace collected
    • Model trained
    • Reran with model policies

Evaluation - YSCB

  • Yahoo! Cloud Serving Benchmark
  • Follows Zipfian distribution (few pages get accessed most)
  • A, F write-heavy

Evaluation - Latency

  • Ranking sees large overhead, protect not as much
  • Overhead is noticeable with larger cache pressure

Evaluation - Twitter Traces

  • Twitter production Cache traces
  • Lot more cache pressure -> ~6-8x ins/sec
  • No heuristic performs best

Reflection, Limitations, Future Work

  • Overfitting - needs train/test separation
    • Hard to fit to changing workloads
  • Demonstrates that neural prediction can be run relatively easily in kernel space
  • Is it possible to create for other kernel components?
    • Task scheduler, prefetch

Thank You!

github.com/JayAndJef

Made with Slides.com