Iliana Fayolle
01 October 2026



Clémentine MAURICE Senior researcher - CNRS Supervisor
Gilles GRIMAUD Full Professor - University of Lille President
Davide BALZAROTTI Full Professor - EURECOM Reviewer
Tamara REZK Senior researcher - Inria Sophia Antipolis Reviewer
Herbert BOS Full Professor - Vrije Universiteit Amsterdam Examiner
Olivier LEVILLAIN Assistant professor - Télécom SudParis Examiner
Walter RUDAMETKIN Full Professor - University of Rennes Guest
Side Channels in Web Environments
From Microarchitectural Attacks to Developer-Centric Countermeasures
Side Channels in Web Environments
From Microarchitectural Attacks to Developer-Centric Countermeasures

Introduction
1 / 44
Context




1 / 44
Introduction
Context




System





Sensitive
data


Side channels: Definition










Leak sources
Computation
1. Collect
2. Deduce
system
Unintended sources of information leakage
sensitive data
" from a that may be exploited to infer ."
2 / 44








Covert channels
Side channels
Introduction




System





Sensitive
data












Leak sources
Computation
1. Collect

2 / 44
Side channels: Definition
Introduction

















Outdated Methodologies

3 / 44
Introduction
Problem 1: Flawed Benchmarking Practices
Are they good proxies to evaluate the attack?
Usual Targets
Fingerprinting
Alexa top 100 websites
Control-Flow (CF)
v1.1.{0, 1}h, and v3.2
GnuPG v1.4.{13, 14}
v1.6.3, and v1.7.5
GnuPG libgcrypt v1.5.{0, 2},
OpenSSL v1.0.{1, 2}e,
Memory Access
WolfSSL v4.2.0, and v5.6.4
MbedTLS v1.3.10, v2.{5, 6, 7,
OpenSSL v1.0.1{e, f}, v0.9.8b,
13, 14, 15}, and v3.{0, 1, 4}
v1.1.0g, and v1.0.2



















Outdated Methodologies
Outdated Methodologies

Unfair Comparisons


3 / 44
Introduction
Problem 1: Flawed Benchmarking Practices




















Outdated Methodologies

Outdated Methodologies

Unfair Comparisons


Reproducibility Issues




3 / 44
Introduction
Problem 1: Flawed Benchmarking Practices



















Outdated Methodologies
Outdated Methodologies

Unfair Comparisons

Reproducibility Issues


How should microarchitectural side-channel attacks be meaningfully evaluated and compared?
3 / 44
Introduction
Problem 1: Flawed Benchmarking Practices
Control flow
Memory usage / allocation






4 / 44
Execution NOT depending on a secret
Secure = Constant-time

Vulnerable
Execution depending on a secret

Introduction
Vulnerable Code Examples
Control flow
Memory usage / allocation









Constant-time control flow
Constant-time Memory usage / allocation



4 / 44
Introduction
No-More-Vulnerable Code Examples
5 / 44


Vulnerable Code



They're not that hard to mitigate: What Cryptographic Library Developers Think About Timing Attacks.
J. Jancar, M. Fourné, D. De Almeida Braga, M. Sabt, P. Schwabe, G. Barthe, P.-A. Fouque, Y. Acar. S&P'22.
"These results must be false": A usability evaluation of constant-time analysis tools.
M. Fourné, D. De Almeida Braga, J. Jancar, M. Sabt, P. Schwabe, G. Barthe, P.-A. Fouque, Y. Acar. USENIX'24.
How can developers be effectively encouraged to write constant time code?
Constant Time Code

Introduction
Problem 2: Unused Security Tools


IP: 20.42.10.176
OS: Android 15
Device screen size: 490 x 1080
Browser: Firefox
Language: en-US
User-Agent (UA): Mozilla/5.0 (Android 15;
Mobile; rv:155.0) Gecko/155.0 Firefox/155.0




IP: 54.20.39.107
OS: Linux x86_64
Device screen size: 1920 × 1080
Browser: Google Chrome
Language: fr-FR
User-Agent (UA): Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/144.0.0.0 Safari/537.36

Side channels
Fingerprinting

Beauty and the Beast: Diverting Modern Web Browsers to Build Unique Browser Fingerprints.
P. Laperdrix, W. Rudametkin, B. Baudry. S&P'16.
6 / 44
Introduction
Fingerprinting: Definition
7 / 44




























How effective are modern defenses, and in particular fingerprinting techniques, against LLM Bots?
Introduction
Problem 3: Unknown Effective Defenses against LLM Bots
How should microarchitectural side-channel attacks be meaningfully evaluated and compared?
How can developers be effectively encouraged to write constant time code?
How effective are modern defenses, and in particular fingerprinting techniques, against LLM Bots?
8 / 44
Research Questions
Benchmarking Practices
Security Tools Adoption
LLM Bots Detection
9 / 44
How should microarchitectural side-channel attacks be meaningfully evaluated and compared?
Benchmarking Practices
How can developers be effectively encouraged to write constant time code?
How effective are modern defenses, and in particular fingerprinting techniques, against LLM Bots?
Security Tools Adoption
LLM Bots Detection
Research Questions
Practice Makes (Im)Perfect: A Look Back at Benchmarking Practices for Microarchitectural Side-Channel Attacks.
I. Fayolle, A. Geimer, D. De Almeida Braga, C. Maurice. CCS'26.
10 / 44
Benchmarking Practices
Context






In what forms do they manifest?
How prevalent are they?
How to avoid them?
Recommendations

Outdated Methodologies
Unfair Comparisons
Reproducibility Issues
Practices
Number of papers per conference over time.
Selected 83 papers
Architecture & Security
Top Rank venues
From 2014 to 2024
More Security papers
Some relevant papers are excluded
More Recent papers


11 / 44
Microarchitectural side- & covert-channel attacks

Benchmarking Practices
Papers Selection



Our Philosophy:
Claim First, Criterion Second.
12 / 44
Benchmarking Practices
Survey Methodology



Our Philosophy:
Claim First, Criterion Second.
12 / 44
Benchmarking Practices
Survey Methodology




Our Philosophy:
Claim First, Criterion Second.
12 / 44
Benchmarking Practices
Survey Methodology

Flaws and their impact on evaluation properties.
Benchmark Omissions
Improper Comparisons
Missing Information
Annotation Labels
Flawed
Incomplete
Unclear
Partially Flawed / Underspecified
Ok
NA
Not Flawed
SoK: Benchmarking Flaws in Systems Security.
E. van der Kouwe, G. Heiser, D. Andriesse, H. Bos, C. Giuffrida. EuroS&P'19.
13 / 44
Benchmarking Practices
Taxonomy of Flaws
Distribution of the number of flaws per paper.
14 / 44
~ 5.5 flaws per paper

Distribution of side-channel attack types.
218 Attacks
71 Covert Channels
147 Side Channels
Benchmarking Practices
Survey Results: General Observations


Summary of benchmarking flaws across all evaluated papers.
15 / 44
Benchmarking Practices
Survey Results

Summary of benchmarking flaws across all evaluated papers.
15 / 44
Benchmarking Practices
Survey Results

Summary of benchmarking flaws across all evaluated papers.
15 / 44
Benchmarking Practices
Survey Results
Security
16 / 44
Benchmarking Practices
Architecture vs. Security Practices
Architecture
>> Flaws
Median number of (partial) flaws per paper across all conferences.

16 / 44
Benchmarking Practices
Architecture vs. Security Practices
>> Evaluated Platforms
Security
>> Flaws
Median number tested platforms per paper across all conferences.
Architecture

Covert channels and side-channel attack types across conferences.

16 / 44
Benchmarking Practices
Architecture vs. Security Practices
Security
>> Covert Channels
>> Flaws
>> Diversity of Side Channels
>> Evaluated Platforms
Architecture
>> Complete Artifacts
16 / 44
Benchmarking Practices
Architecture vs. Security Practices
>> Covert Channels
>> Diversity of Side Channels
>> Evaluated Platforms
Security
>> Flaws
Both
Security
Availability of artifacts over time.
Architecture



Architecture
The dashed lines show when artefact evaluation began.
>> (Incomplete) Artifacts
17 / 44
Benchmarking Practices
Practices Evolution over Time
Both
Security
Availability of artifacts over time.
The dashed lines show when artefact evaluation began.
Architecture



>> (Incomplete) Artifacts
>> Side-Channel Evaluations
17 / 44
Benchmarking Practices
Practices Evolution over Time
Evolution of median number of benchmarks per paper over time
across all conferences.

18 / 44
Not All Those Who Share Are Lost: Analyzing 25 Years of Cybersecurity Artifact Sharing Practices Through Automated Discovery.
D. Vansteenhuyse, A. Bols, L. Desmet, V. Le Pochat, J. Van Bulck, M. Bognar. USENIX'26.
Reproducibility in Applied Security Conferences: An 11-Year Review on Artifacts and Evaluation Committees.
D. Olszewski, A. Lu, A. Crowder, N. Bennett, S. Layton, S. Hrushikesh V. Bhupathiraju, T. Tucker, S. Kalgutkar, H. Ver Helst, C. Stillman, K. R. B. Butler,
S. Rampazzi, P. Traynor. ACM REP'25.
Lessons Learned from Five Years of Artifact Evaluations at EuroSys.
D. C. D’Elia, T. D. Doudali, C. Giuffrida, M. Matos, M. Payer, S. Pirelli, G. Portokalidis, V. Schiavoni, S. Signorello, A. Vahldiek-Oberwagner. ACM REP'25.
Reviewers

Benchmarking Practices
Recommendations
Share artifacts
Authors


Community
Standardize artifact evaluation
18 / 44

Invisible Probe: Timing Attacks with PCIe Congestion Side-channel.
M. Tan, J. Wan, Z. Zhou, Z. Li. S&P'21.
Reviewers

Benchmarking Practices
Recommendations
Share artifacts
Authors


Community
Standardize artifact evaluation
18 / 44

RELOAD+REFRESH: Abusing Cache Replacement Policies to Perform Stealthy Cache Attacks.
S. Briongos, P. Malagón, J. M. Moya, T. Eisenbarth. USENIX'20.
Smallest distinguishable unit and its observation mapping.
Spatial Resolution
Reviewers

Benchmarking Practices
Recommendations
Share artifacts
Prioritize resolution metrics
Authors


Community
Standardize artifact evaluation
Promote comprehensive benchmarking practices
18 / 44
Cache Template Attacks: Automating Attacks on Inclusive Last-Level Caches.
D. Gruss, R. Spreitzer, S. Mangard. USENIX'15.

Smallest distinguishable unit and its observation mapping.
Spatial Resolution
Reviewers

Benchmarking Practices
Recommendations
Share artifacts
Prioritize resolution metrics
Authors


Community
Standardize artifact evaluation
Promote comprehensive benchmarking practices
18 / 44
Temporal Resolution
Time per sample and maximum sampling rate.
TunneLs for Bootlegging: Fully Reverse-Engineering GPU TLBs for Challenging Isolation Guarantees of NVIDIA MIG.
Z. Zhang, T. N. Allen, F. Yao, X. Gao, R. Ge. CCS'23.

Reviewers

Benchmarking Practices
Recommendations
Share artifacts
Prioritize resolution metrics
Authors


Community
Standardize artifact evaluation
Promote comprehensive benchmarking practices
18 / 44
Temporal Resolution
Time per sample and maximum sampling rate.
POWERT Channels: A Novel Class of Covert Communication Exploiting Power Management Vulnerabilities.
S. K. Khatamifard, L. Wang, A. Das, S. Köse, U. R. Karpuzcu. HPCA'19.

Reviewers

Benchmarking Practices
Recommendations
Share artifacts
Prioritize resolution metrics
Authors


Community
Standardize artifact evaluation
Promote comprehensive benchmarking practices
18 / 44
Reviewers

Demand relevant benchmarks
Focus on core properties
Benchmarking Practices
Recommendations
Share artifacts
Prioritize resolution metrics
Authors


Community
Standardize artifact evaluation
Promote comprehensive benchmarking practices
19 / 44
Share artifacts
Prioritize resolution metrics
Authors


Reviewers

Community
Demand relevant benchmarks
Focus on core properties
Standardize artifact evaluation
Promote comprehensive benchmarking practices
How should microarchitectural side-channel attacks be meaningfully evaluated and compared?
Benchmarking Practices
Takeaway


Practices (resolution metrics...)

Outdated Methodologies
Unfair Comparisons
Reproducibility Issues
Practices




Research Questions
How can developers be effectively encouraged to write constant time code?
Security Tools Adoption
How should microarchitectural side-channel attacks be meaningfully evaluated and compared?
Benchmarking Practices
How effective are modern defenses, and in particular fingerprinting techniques, against LLM Bots?
LLM Bots Detection
20 / 44
Semi-Automated and Easily Interpretable Side-Channel Analysis for Modern JavaScript.
I. Fayolle, J. Wichelmann, A. Köhl, W. Rudametkin, T. Eisenbarth, C. Maurice. CANS'24.
21 / 44
Control flow
Memory usage / allocation




Vulnerable Code


Control flow
Memory usage / allocation



Constant Time Code




Security Tools Adoption
Context
CacheAudit
Binsec/Rel
ctgrind
CacheQL
DiffFuzz
ct-fuzz
Microwalk
Vulnerability Analysis Tools
...






Stack Overflow developer survey 2025. Stack Overflow. https://survey.stackoverflow.co/2025/, accessed: 09-14-2026.
Python developers survey 2024 results. jetbrains. https://lp.jetbrains.com/python-developers-survey-2024/, accessed: 09-14-2026.
Not integrated into development processes
Compatibility problems
Not well documented
Poor usability
Problems
Security Tools Adoption
Context
21 / 44
They're not that hard to mitigate: What Cryptographic Library Developers Think About Timing Attacks.
J. Jancar, M. Fourné, D. De Almeida Braga, M. Sabt, P. Schwabe, G. Barthe, P.-A. Fouque, Y. Acar. S&P'22.
"These results must be false": A usability evaluation of constant-time analysis tools.
M. Fourné, D. De Almeida Braga, J. Jancar, M. Sabt, P. Schwabe, G. Barthe, P.-A. Fouque, Y. Acar. USENIX'24.



Analyses during Continuous Integration (CI)

Statistical + Dynamic Analysis

Quickly Locates and Quantifies Leakages

Microwalk tool: https://github.com/microwalk-project/Microwalk
Microwalk-CI: Practical Side-Channel Analysis for JavaScript Applications.
J. Wichelmann, F. Sieck, A. Pätschke, T. Eisenbarth. CCS'22.
Microwalk: A Framework for Finding Side Channels in Binaries.
J. Wichelmann, A. Moghimi, T. Eisenbarth, B. Sunar. ACSAC'18.
22 / 44
Security Tools Adoption
Microwalk-CI: Presentation


Will their execution paths be different?

YES


YES
23 / 44
Security Tools Adoption
Microwalk-CI: Workflow

Modern cannot be analyzed

Post-2015 ECMAScript Versions
Deferred Execution
ES Modules (ESM)
Classes
Asynchronous Programming
Jalangi2

Compatibility Issue
Limits
24 / 44
Security Tools Adoption
Microwalk-CI: Limits
Compatibility Issue
Limits
Tedious Test Writing

Writing tests is tedious and error-prone



Security Tools Adoption
Microwalk-CI: Limits
24 / 44
Compatibility Issue
Limits
Tedious Test Writing
Limited Report Interpretability




The final report is both impractical and incomplete
Security Tools Adoption
Microwalk-CI: Limits
24 / 44
Compatibility Issue
Limits
Tedious Test Writing
Limited Report Interpretability
Solutions
Actionable Final Report
Semi-Automated Test Writing Process
Modern JavaScript Support
Security Tools Adoption
Microwalk-CI: Limits
24 / 44

Excerpt from the aes-js analysis target.
Automated
Test Case Generation
Subtargets Introduction
25 / 44
Security Tools Adoption
Semi-Automated Test Writing Process



code-coverage

code-coverage
Leak Information Box
Detailed Header
Extract of the HTML report after analyzing the @noble/ciphers library.



26 / 44
Security Tools Adoption
Actionable Final Report
Vulnerabilities

Performance

Development Effort
Test on Modern

Old vs. New
aes-js
node-forge
noble@ciphers noble@curves
Maximize code-coverage

Replicate Wichelmann CCS'22
Find more leakages
Identify missed code path


Support Modern

Perform better
135s vs. 73s ; < 2GB

80%
45%
93%
72%
Need 1 file per library
464 lines vs. 206 lines

27 / 44
Security Tools Adoption
Evaluation
Find more vulnerabilities with less effort
Compatibility Issue
Limits
Tedious Test Writing
Limited Report Interpretability
Solutions
Actionable Final Report
Semi-Automated Test Writing Process
Modern JavaScript Support
Wider adoption requires improved usability
How can developers be effectively encouraged to write constant time code?
28 / 44
Security Tools Adoption
Takeaway
Research Questions
How effective are modern defenses, and in particular fingerprinting techniques, against LLM Bots?
LLM Bots Detection
How should microarchitectural side-channel attacks be meaningfully evaluated and compared?
Benchmarking Practices
How can developers be effectively encouraged to write constant time code?
Security Tools Adoption
29 / 44
On the Internet, Nobody Knows You’re an LLM Bot: Unmasking Web Agents with Multi-Layer Fingerprinting.
I. Fayolle, S. Bouhenniche, S. Pélissier, P. Laperdrix, C. Maurice, W. Rudametkin. arXiv, 2026. (Under Submission)

Global Internet Traffic for the past 10 years.
Bad bot report: the rapid rise of bots and the unseen risk for business. Imperva. 2025.
https://www.imperva.com/resources/wp-content/uploads/sites/6/reports/2025-Bad-Bot-Report.pdf, accessed: 09-14-2026.
30 / 44
LLM Bots Detection
Context





HTTP-Based Scrapers
Large-Scale Data Extraction
Execute JavaScript
cURL wget scrapy
LLM-Based Chatbot
Browser Automation
Framework (BAF)
Execute JavaScript
Selenium Puppeteer Playwright
Scripted Interactions
Control Web Browser
Web Agents
Natural Language Instructions
BAF + LLM-Based Reasonning
Large-Scale High Value Data Extraction
OpenClaw Browser Use Skyvern ChatGPT Agent ...
Integrate Stealth Mechanisms








31 / 44
LLM Bots Detection
Taxonomy of Bots






IP: 20.42.10.176
UA: GPTBOT
*************
*****


IP address reputation
Rule-based detection



32 / 44
LLM Bots Detection
Anti-Bot Mechanisms






IP address reputation
Rule-based detection



LLM Bots Detection
Anti-Bot Mechanisms



32 / 44

IP: 20.42.10.176
UA: Mozilla/5.0
(iPhone14,2; *******)
*****



CAPTCHAs






IP address reputation
Rule-based detection



LLM Bots Detection
Anti-Bot Mechanisms



32 / 44
CAPTCHAs
Proof-of-work







IP address reputation
Rule-based detection


LLM Bots Detection
Anti-Bot Mechanisms



32 / 44
CAPTCHAs
Proof-of-work
Fingerprinting



Screen width: 1920;
Screen height: 1080;
Timezone: UTC+02:00;
|
|
|

IP: 20.42.10.176
UA: Mozilla/5.0
(iPhone14,2; *******)
*****







IP address reputation
Rule-based detection



LLM Bots Detection
Anti-Bot Mechanisms



32 / 44
IP address reputation
Rule-based detection
CAPTCHAs
Proof-of-work
Fingerprinting
Proprietary Solutions















LLM Bots Detection
Anti-Bot Mechanisms



32 / 44
33 / 44
LLM Bots Detection
Overview

IP address reputation
Rule-based detection
CAPTCHAs
Proof-of-work
Fingerprinting
Proprietary Solutions







Can discriminative attributes uniquely identify bots?






Scrapers


Web Agents
BAF
Do bots pass defenses?






Data collection infrastructure for the deployment and instrumentation of honeysites.
LLM Bots Detection
Experimental Setup
34 / 44

Data collection infrastructure for the deployment and instrumentation of honeysites.
LLM Bots Detection
Experimental Setup
34 / 44

IP address source port

IP address User-Agent
cookie_id
DateTime
IP address cookie_id User-Agent DateTime
source port
Data collection infrastructure for the deployment and instrumentation of honeysites.
IP address User-Agent
cookie_id
DateTime
IP address cookie_id User-Agent DateTime
source port
LLM Bots Detection
Experimental Setup
34 / 44

IP address source port


IP address User-Agent
cookie_id
DateTime
IP address cookie_id User-Agent DateTime
source port
Data collection infrastructure for the deployment and instrumentation of honeysites.
LLM Bots Detection
Experimental Setup
34 / 44










Subjects
Selenium, Puppeteer, Playwright
Local vs. Cloud
Open- vs. Close-Source
Evasion Capabilities
Characteristics
LLM Bots Detection
Methodology
OpenClaw,
ChatGPT Agent,
Skyvern,
Browser Use,
Crawl4AI,
Claude for Chrome
Chrome v144.0.7559.96, Firefox v136.0, and v147.0.1
cURL, wget, scrapy
35 / 44










Subjects

Honeysite appearance: baseline site.
Prompt.

LLM Bots Detection
Methodology
35 / 44










Subjects





Anti-Bot Mechanism
Timeout
Blocked
Passed


Honeysite appearance: baseline site.
LLM Bots Detection
Methodology
35 / 44
IP
TLS
Browser Fingerprint
Prompt output











Subjects
Defense passed?













Visit
1,449 Initiated Visits
1,383 Visits with fingerprints correctly linked
From 01-28 to 02-24-2026


LLM Bots Detection
Dataset
36 / 44

Bot access to honeysites under different anti-bot mechanisms.
Takeaway
JavaScript-based defense are effective against


LLM Bots Detection
Evaluating Anti-Bot Mechanisms
37 / 44

Takeaway
JavaScript-based defense are effective against


struggles with interactive challenges




Bot access to honeysites under different anti-bot mechanisms.
LLM Bots Detection
Evaluating Anti-Bot Mechanisms
37 / 44

Takeaway
JavaScript-based defense are effective against


struggles with interactive challenges






Some bypass all defense
Bot access to honeysites under different anti-bot mechanisms.
LLM Bots Detection
Evaluating Anti-Bot Mechanisms
37 / 44

Takeaway
JavaScript-based defense are effective against


struggles with interactive challenges






Some bypass all defense
Stealth mode may increase detectability
Bot access to honeysites under different anti-bot mechanisms.
LLM Bots Detection
Evaluating Anti-Bot Mechanisms
37 / 44
IP
TLS
Browser Fingerprint
LLM Bots Detection
Characterizing Bots
Takeaway
Fixed execution environments
stable fingerprints
Stealth mechanisms
inconsistencies





Cloud-Based bots & easier to detect
Local bots inherit the user's trusted IP address
Cloud can connect through:
Datacenter infrastructure ;
Residential proxies


Discriminative Attributes
User-Agent ; Screen Resolution/Display ; Permissions State ; Cookie Management ; Number of CPU Cores ; Stealth Behavior ; Tool-Specific Attributes
IP
TLS Fingerprint
38 / 44
LLM Bots Detection
Multi-Layer Bots Classification
Classification Evaluation using Random Forest


(with alea ~7%).
Can discriminative attributes uniquely identify bots? 14 classes (Human, Chatgpt Agent, Browser Use Local, ...)
Discriminative Attributes
User-Agent ; Screen Resolution/Display ; Permissions State ; Cookie Management ; Number of CPU Cores ; Stealth Behavior ; Tool-Specific Attributes
IP
TLS Fingerprint
39 / 44
Takeaway
Individual fingerprinting =
Combined fingerprinting =





Stealth Mode




IP
TLS
Browser
...
...
...
Hard Challenges
LLM Bots Detection
Takeaway
How effective are modern defenses, and in particular fingerprinting techniques, against LLM Bots?
40 / 44
Conclusion

Side Channels in Web Environments
From Microarchitectural Attacks to Developer-Centric Countermeasures


IP
TLS
Browser
Hard Challenges
...
...

Find more vulnerabilities with less effort
Vulnerable Code



Low level
Microarchitecture
High level
Web-based applications & systems
41 / 44






Practices
Recommendations
Conclusion



IP
TLS
Browser
Hard Challenges
...
...

Find more vulnerabilities with less effort
Vulnerable Code



Low level
Microarchitecture
High level
Web-based applications & systems
New Benchmark
Extend Microwalk
New defense
What's next?
42 / 44
Side Channels in Web Environments
From Microarchitectural Attacks to Developer-Centric Countermeasures






Practices
Recommendations
Conclusion






The Challenges of Bringing Cryptography from Research Papers to Products: Results from an Interview Study with Experts.
K. Fischer, I. Trummová, P. Gajland, Y. Acar, S. Fahl, M. A.Sasse. USENIX'24.
LLMs Cannot Reliably Identify and Reason About Security Vulnerabilities (Yet?): A Comprehensive Evaluation, Framework,
and Benchmarks. S. Ullah, M. Han, S. Pujar, H. Pearce, A. K. Coskun, G. Stringhini. S&P'24.
In generative AI we trust: can chatbots effectively verify political information?
E. Kuznetsova, M. Makhortykh, V. Vziatysheva, M. Stolze, A. Baghumyan, A. Urman. Journal of Computational Social Science, 2025.
What News Do People Get on Social Media? Analyzing Exposure and Consumption of News through Data Donations.
S. Chouaki, A. Chakraborty, O. Goga, S. Zannettou, WWW'24.




43 / 44
How should microarchitectural side-channel attacks be meaningfully evaluated and compared?
How can developers be effectively encouraged to write constant time code?
How effective are modern defenses, and in particular fingerprinting techniques, against LLM Bots?
Summary
Benchmarking Practices
Security Tools Adoption
LLM Bots Detection
On the Internet, Nobody Knows You’re an LLM Bot: Unmasking Web Agents with Multi-Layer Fingerprinting.
I. Fayolle, S. Bouhenniche, S. Pélissier, P. Laperdrix, C. Maurice, W. Rudametkin. arXiv, 2026. (Under Submission)
Semi-Automated and Easily Interpretable Side-Channel Analysis for Modern JavaScript.
I. Fayolle, J. Wichelmann, A. Köhl, W. Rudametkin, T. Eisenbarth, C. Maurice. CANS'24.
Practice Makes (Im)Perfect: A Look Back at Benchmarking Practices for Microarchitectural Side-Channel Attacks.
I. Fayolle, A. Geimer, D. De Almeida Braga, C. Maurice. CCS'26.
44 / 44

Flaws and their impact on evaluation pro.
Backup
Benchmarking Practices
Backup
Benchmarking Practices
Summary of benchmarking flaws across all evaluated papers.

Backup
Benchmarking Practices
Prevalence of the 19 flaws in the 83 surveyed papers



Fig. 4: Distribution of reported properties in covert-channel evaluations.
Fig. 5: Distribution of side-channel attack types.
59 (Speed)
30 (Speed, Error Rate)
Reported Metrics per Paper
18 (Speed, Error Rate, ...)
218 Attacks
71 Covert Channels
147 Side Channels
Attacked Platforms per Paper
73 CPUs (and 65 Intel)
9 GPUs (and 3 iGPUs)
9 TEEs
Backup
Benchmarking Practices
Security
Architecture
>> Flaws


Number of flaws per paper over time.
Security
Architecture
Backup
Benchmarking Practices
Benchmarked platforms per paper over time across all conferences.

>> Evaluated Platforms
Security
Architecture
>> Flaws
Backup
Benchmarking Practices
Number of evaluations per paper over time across all conferences.

Backup
Benchmarking Practices

Reported properties in covert-channel evaluations across conference.
Backup
Benchmarking Practices
Ensure fair comparisons
Define & report errors clearly
Test cross-platform generalization
Be explicit about evaluation details
Evaluate every claim empirically
Share artifacts
Prioritize resolution metrics






Authors

Backup
Benchmarking Practices






Authors

Avoid requiring full crypto attacks
Push for realistic setups
Demand relevant benchmarks
Focus on core properties
Reviewers





Backup
Benchmarking Practices






Authors

Encourage meaningful metrics
Foster robust benchmarking culture
Standardize artifact evaluation
Promote complete reporting
Community







Reviewers





Backup
Benchmarking Practices

Trace generation pipeline. The red arrows denote execution flow.



Modern

Backwards-Compatible JavaScript



Instrumentation
Backup
Security Tools Adoption

Comparison between Microwalk V3.2.0 and Microwalk using the enhanced analysis
toolchain on aes-js and node-forge libraries.
Backup
Security Tools Adoption

Results of the enhanced analysis toolchain with 2 modern JavaScript
cryptographic libraries: @noble/ciphers ; @noble/curves.
Backup
Security Tools Adoption
Feature comparison of selected Web Agents.







Backup
LLM Bots Detection

Normal Prompt (i) and Special Prompt (ii).
Backup
LLM Bots Detection
1,449 Initiated Visits
1,383 Visits with fingerprints correctly linked
From 01-28 to 02-24-2026



Collected visits per tool.
Backup
LLM Bots Detection

Bot access to honeysites under different anti-bot mechanisms.
Takeaway
JavaScript-based defense are effective against


struggles with interactive challenges






Some bypass all defense
Stealth mode may increase detectability
Backup
LLM Bots Detection
IP


Takeaway
Local tools inherit the user's trusted IP address
Cloud can connect through:


1. Datacenter infrastructure
2. Residential proxies
IP-layer characteristics per tool.
Backup
LLM Bots Detection
Backup
LLM Bots Detection


TLS Client Hello Packet obtained with Wireshark
Mapping between JA4 index, full JA4 and its features.
Backup
LLM Bots Detection

IP
TLS

JA4 Variability
With pre_shared_key (PSK) or not
QUIC vs. TCP
Takeaway
Cloud-Based bots & are easier to detect
Stealth mode has no effect on JA4


Dominant JA4 and discriminative power per tool.
Backup
LLM Bots Detection
: Attribute
: Tools
: Value
A-Score
V-Score
Inter-tool Value Exclusivity
Intra-tool Value Probability
How stable and unique an attribute value is for a tool ?
How stable and unique an attribute is for a tool ?
Backup
LLM Bots Detection
Discriminative Attributes
User-Agent
ChatGPT Agent:
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/141.0.0.0 Safari/537.36
Skyvern:
Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/144.0.0.0 Safari/537.36 Edg/144.0.0.0
IP
TLS
Browser Fingerprint
Backup
LLM Bots Detection
Discriminative Attributes
User-Agent
Screen Resolution and Display
ChatGPT Agent:
1280 × 960
Crawl4AI:
1080 × 600
IP
TLS
Browser Fingerprint
Backup
LLM Bots Detection
Discriminative Attributes
User-Agent
Screen Resolution and Display
Permissions State
ChatGPT Agent & Crawl4AI:
Not Supported
BrowserUse:
Denied
Skyvern & Claude:
Prompt or Granted
IP
TLS
Browser Fingerprint
Backup
LLM Bots Detection
Discriminative Attributes
User-Agent
Screen Resolution and Display
Permissions State
Cookie Management
Claude:
Share Cookie with Human
ChatGPT Agent, Skyvern & OpenClaw:
Reused Cookie
BrowserUse:
New Cookie
IP
TLS
Browser Fingerprint
Backup
LLM Bots Detection
Discriminative Attributes
User-Agent
Screen Resolution and Display
Permissions State
Cookie Management
Number of CPU Cores
Skyvern:
32
ChatGPT Agent:
13
BrowserUse:
From 2 to 64
IP
TLS
Browser Fingerprint
Backup
LLM Bots Detection
Discriminative Attributes
User-Agent
Screen Resolution and Display
Permissions State
Cookie Management
Number of CPU Cores
Stealth Behavior
Crawl4AI Stealth:
Modify User-Agent
Inject synthetic referer
IP
TLS
Browser Fingerprint
Backup
LLM Bots Detection
IP
TLS
Browser Fingerprint
Discriminative Attributes
User-Agent
Screen Resolution and Display
Permissions State
Cookie Management
Number of CPU Cores
Tool-Specific Attributes
Stealth Behavior
ChatGPT Agent:
signature & signature-agent
Crawl4AI:
"default" for the notification permission
Backup
LLM Bots Detection
Setup
Random Forest, XGBoost, CatBoostClassifier
1370 Fingerprints ( )
80% Train (SMOTE) / 20% Test
Run 10 times with random seeds




Classification Evaluation using Random Forest.
Backup
LLM Bots Detection

Confusion matrices for individual fingerprinting layers with Random Forest.
Backup
LLM Bots Detection
Confusion matrices for combined fingerprinting layers with Random Forest.

Backup
LLM Bots Detection
PhD Thesis Defense
By Iliana Fayolle
PhD Thesis Defense
- 20