Software Engineering PhD Day
09/12/25
SoK: Nobody’s Perfect:
A Look Back At Microarchitectural Side-Channel Attacks Benchmarking
Iliana FAYOLLE
Antoine GEIMER
Daniel DE ALMEIDA BRAGA
Clémentine MAURICE
Iliana FAYOLLE
Antoine GEIMER
Daniel DE ALMEIDA BRAGA
Clémentine MAURICE
SoK: Nobody’s Perfect:
A Look Back At Microarchitectural Side-Channel Attacks Benchmarking
Context
System
Computation
Measurements
Attacker
(secret) input
(secret) output
?
?
SoK: Nobody’s Perfect:
A Look Back At Microarchitectural Side-Channel Attacks Benchmarking
Context
Outdated Benchmarks
Non-Comparable Results
Inconsistent Metrics
Low Reproducibility
Motivation
Systematic study
- 77 papers
Taxonomy of flaws
Recommendations
Methodology
SoK: Nobody’s Perfect:
A Look Back At Microarchitectural Side-Channel Attacks Benchmarking
Results
~ 5.5 flaws per paper
SoK: Nobody’s Perfect:
A Look Back At Microarchitectural Side-Channel Attacks Benchmarking
Recommendations
Share artifacts
Prioritize resolution metrics
Ensure fair comparisons
Define & report errors clearly
Test cross-platform generalization
Be explicit about evaluation details
Evaluate every claim empirically
Share artifacts
Prioritize resolution metrics
SoK: Nobody’s Perfect:
A Look Back At Microarchitectural Side-Channel Attacks Benchmarking
Takeways
Paper here :)
No more…
Proxy benchmarks
Inconsistent metrics
Hard-to-reproduce experiments
And more…
Shared artifacts & explicit evaluation details
Resolution-driven benchmarking
Fair, realistic, cross-platform comparisons
SoK: Nobody’s Perfect:
A Look Back At Microarchitectural Side-Channel Attacks Benchmarking
Number of Papers per Conference per Year
SoK: Nobody’s Perfect:
A Look Back At Microarchitectural Side-Channel Attacks Benchmarking
List of Flaws and their Impact on the Benchmark Quality
SoK: Nobody’s Perfect:
A Look Back At Microarchitectural Side-Channel Attacks Benchmarking
Full Recommendations
|
Authors |
Reviewers & PCs |
Community |
|---|---|---|
| • Share artifacts (code, configs) • Prioritize resolution metrics • Ensure fair comparisons • Define and report errors clearly • Test cross-platform generalization • Be explicit about all eval details • Evaluate every claim empirically |
• Demand relevant benchmarks • Focus on core properties (resolution, noise) • Avoid requiring full crypto attacks • Push for realistic setups |
• Standardize artifact evaluation • Promote complete reporting • Encourage meaningful metrics • Foster robust benchmarking culture |