Iliana Fayolle

01 October 2026

Clémentine MAURICE                                  Senior researcher - CNRS Supervisor

Gilles GRIMAUD                                 Full Professor - University of Lille President

Davide BALZAROTTI                                       Full Professor - EURECOM Reviewer

Tamara REZK                         Senior researcher - Inria Sophia Antipolis Reviewer

Herbert BOS                   Full Professor - Vrije Universiteit Amsterdam Examiner

Olivier LEVILLAIN                    Assistant professor - Télécom SudParis Examiner

Walter RUDAMETKIN                        Full Professor - University of Rennes Guest

Side Channels in Web Environments

From Microarchitectural Attacks to Developer-Centric Countermeasures

Side Channels in Web Environments

From Microarchitectural Attacks to Developer-Centric Countermeasures

Introduction

1 / 44

Context

1 / 44

Introduction

Context

System     

Sensitive

data

Side channels: Definition

Leak sources

Computation

1. Collect

2. Deduce

system

 

  Unintended sources of information leakage                          

                   sensitive data

 

 "                                                                                 from a                that may be    exploited to infer                          ."

2 / 44

Covert channels

Side channels

Introduction

System     

Sensitive

data

Leak sources

Computation

1. Collect

2 / 44

Side channels: Definition

Introduction

Outdated Methodologies

3 / 44

Introduction

Problem 1: Flawed Benchmarking Practices

Are they good proxies to evaluate the attack?

Usual Targets

Fingerprinting

Alexa top 100 websites

Control-Flow (CF)

v1.1.{0, 1}h, and v3.2

GnuPG v1.4.{13, 14}

v1.6.3, and v1.7.5

GnuPG libgcrypt v1.5.{0, 2},

OpenSSL v1.0.{1, 2}e,

Memory Access

WolfSSL v4.2.0, and v5.6.4

MbedTLS v1.3.10, v2.{5, 6, 7,

OpenSSL v1.0.1{e, f}, v0.9.8b,

13, 14, 15}, and v3.{0, 1, 4}

v1.1.0g, and v1.0.2

Outdated Methodologies

Outdated Methodologies

Unfair Comparisons

3 / 44

Introduction

Problem 1: Flawed Benchmarking Practices

Outdated Methodologies

Outdated Methodologies

Unfair Comparisons

Reproducibility Issues

3 / 44

Introduction

Problem 1: Flawed Benchmarking Practices

Outdated Methodologies

Outdated Methodologies

Unfair Comparisons

Reproducibility Issues

How should microarchitectural side-channel attacks be meaningfully evaluated and compared?

3 / 44

Introduction

Problem 1: Flawed Benchmarking Practices

Control flow

Memory usage / allocation

4 / 44

Execution NOT depending on a secret

Secure = Constant-time

Vulnerable

Execution depending on a secret

Introduction

Vulnerable Code Examples

Control flow

Memory usage / allocation

Constant-time control flow

Constant-time Memory usage / allocation

4 / 44

Introduction

No-More-Vulnerable Code Examples

5 / 44

Vulnerable Code

They're not that hard to mitigate: What Cryptographic Library Developers Think About Timing Attacks.

J. Jancar, M. Fourné, D. De Almeida Braga, M. Sabt, P. Schwabe, G. Barthe, P.-A. Fouque, Y. Acar. S&P'22.

"These results must be false": A usability evaluation of constant-time analysis tools.

M. Fourné, D. De Almeida Braga, J. Jancar, M. Sabt, P. Schwabe, G. Barthe, P.-A. Fouque, Y. Acar. USENIX'24.

How can developers be effectively encouraged to write constant time code?

Constant Time Code

Introduction

Problem 2: Unused Security Tools

IP: 20.42.10.176

OS: Android 15

Device screen size: 490 x 1080

Browser: Firefox

Language: en-US

User-Agent (UA): Mozilla/5.0 (Android 15;

Mobile; rv:155.0) Gecko/155.0 Firefox/155.0

IP: 54.20.39.107

OS: Linux x86_64

Device screen size: 1920 × 1080

Browser: Google Chrome

Language: fr-FR

User-Agent (UA): Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/144.0.0.0 Safari/537.36

Side channels

Fingerprinting

Beauty and the Beast: Diverting Modern Web Browsers to Build Unique Browser Fingerprints.

P. Laperdrix, W. Rudametkin, B. Baudry. S&P'16.

6 / 44

Introduction

Fingerprinting: Definition

7 / 44

How effective are modern defenses, and in particular fingerprinting techniques, against LLM Bots?

Introduction

Problem 3: Unknown Effective Defenses against LLM Bots

How should microarchitectural side-channel attacks be meaningfully evaluated and compared?

How can developers be effectively encouraged to write constant time code?

How effective are modern defenses, and in particular fingerprinting techniques, against LLM Bots?

8 / 44

Research Questions

Benchmarking Practices

Security Tools Adoption

LLM Bots Detection

9 / 44

How should microarchitectural side-channel attacks be meaningfully evaluated and compared?

Benchmarking Practices

How can developers be effectively encouraged to write constant time code?

How effective are modern defenses, and in particular fingerprinting techniques, against LLM Bots?

Security Tools Adoption

LLM Bots Detection

Research Questions

Practice Makes (Im)Perfect: A Look Back at Benchmarking Practices for Microarchitectural Side-Channel Attacks.

I. Fayolle, A. Geimer, D. De Almeida Braga, C. Maurice. CCS'26.

10 / 44

Benchmarking Practices

Context

In what forms do they manifest?

How prevalent are they?

How to avoid them?

Recommendations

Outdated Methodologies

Unfair Comparisons

Reproducibility Issues

Practices

   Number of papers per conference over time.

Selected 83 papers

 Architecture & Security

Top Rank venues

 

From 2014 to 2024

 

More Security papers

Some relevant papers are excluded

More Recent papers

11 / 44

 

       Microarchitectural side- & covert-channel attacks

 

Benchmarking Practices

Papers Selection

Our Philosophy:

Claim First, Criterion Second.

12 / 44

Benchmarking Practices

Survey Methodology

Our Philosophy:

Claim First, Criterion Second.

12 / 44

Benchmarking Practices

Survey Methodology

Our Philosophy:

Claim First, Criterion Second.

12 / 44

Benchmarking Practices

Survey Methodology

Flaws and their impact on evaluation properties.

Benchmark Omissions

Improper Comparisons

Missing Information

Annotation Labels

Flawed

Incomplete

Unclear

Partially Flawed / Underspecified

Ok

NA

Not Flawed

SoK: Benchmarking Flaws in Systems Security.

E. van der Kouwe, G. Heiser, D. Andriesse, H. Bos, C. Giuffrida. EuroS&P'19.

13 / 44

Benchmarking Practices

Taxonomy of Flaws

Distribution of the number of flaws per paper.

14 / 44

~ 5.5 flaws per paper

Distribution of side-channel attack types.

218 Attacks

71 Covert Channels

147 Side Channels

Benchmarking Practices

Survey Results: General Observations

Summary of benchmarking flaws across all evaluated papers.

15 / 44

Benchmarking Practices

Survey Results

Summary of benchmarking flaws across all evaluated papers.

15 / 44

Benchmarking Practices

Survey Results

Summary of benchmarking flaws across all evaluated papers.

15 / 44

Benchmarking Practices

Survey Results

Security

16 / 44

Benchmarking Practices

Architecture vs. Security Practices

Architecture

>> Flaws

Median number of (partial) flaws per paper across all conferences.

16 / 44

Benchmarking Practices

Architecture vs. Security Practices

>> Evaluated Platforms

Security

>> Flaws

Median number tested platforms per paper across all conferences.

Architecture

Covert channels and side-channel attack types across conferences.

16 / 44

Benchmarking Practices

Architecture vs. Security Practices

Security

>> Covert Channels

>> Flaws

>> Diversity of Side Channels

>> Evaluated Platforms

Architecture

>> Complete Artifacts

16 / 44

Benchmarking Practices

Architecture vs. Security Practices

>> Covert Channels

>> Diversity of Side Channels

>> Evaluated Platforms

Security

>> Flaws

Both

Security

Availability of artifacts over time.

Architecture

Architecture

The dashed lines show when artefact evaluation began.

>> (Incomplete) Artifacts

17 / 44

Benchmarking Practices

Practices Evolution over Time

Both

Security

Availability of artifacts over time.

The dashed lines show when artefact evaluation began.

Architecture

>> (Incomplete) Artifacts

>> Side-Channel Evaluations

17 / 44

Benchmarking Practices

Practices Evolution over Time

Evolution of median number of benchmarks per paper over time

across all conferences.

18 / 44

Not All Those Who Share Are Lost: Analyzing 25 Years of Cybersecurity Artifact Sharing Practices Through Automated Discovery.

D. Vansteenhuyse, A. Bols, L. Desmet, V. Le Pochat, J. Van Bulck, M. Bognar. USENIX'26.

Reproducibility in Applied Security Conferences: An 11-Year Review on Artifacts and Evaluation Committees.

D. Olszewski, A. Lu, A. Crowder, N. Bennett, S. Layton, S. Hrushikesh V. Bhupathiraju, T. Tucker, S. Kalgutkar, H. Ver Helst, C. Stillman, K. R. B. Butler,

S. Rampazzi, P. Traynor. ACM REP'25.

Lessons Learned from Five Years of Artifact Evaluations at EuroSys.

D. C. D’Elia, T. D. Doudali, C. Giuffrida, M. Matos, M. Payer, S. Pirelli, G. Portokalidis, V. Schiavoni, S. Signorello, A. Vahldiek-Oberwagner. ACM REP'25.

Reviewers

Benchmarking Practices

Recommendations

Share artifacts

Authors

Community

Standardize artifact evaluation

18 / 44

Invisible Probe: Timing Attacks with PCIe Congestion Side-channel.

M. Tan, J. Wan, Z. Zhou, Z. Li. S&P'21.

Reviewers

Benchmarking Practices

Recommendations

Share artifacts

Authors

Community

Standardize artifact evaluation

18 / 44

RELOAD+REFRESH: Abusing Cache Replacement Policies to Perform Stealthy Cache Attacks.

S. Briongos, P. Malagón, J. M. Moya, T. Eisenbarth. USENIX'20.

Smallest distinguishable unit and its observation mapping.

Spatial Resolution

Reviewers

Benchmarking Practices

Recommendations

Share artifacts

Prioritize resolution metrics

Authors

Community

Standardize artifact evaluation

Promote comprehensive benchmarking practices

18 / 44

Cache Template Attacks: Automating Attacks on Inclusive Last-Level Caches.

D. Gruss, R. Spreitzer, S. Mangard. USENIX'15.

Smallest distinguishable unit and its observation mapping.

Spatial Resolution

Reviewers

Benchmarking Practices

Recommendations

Share artifacts

Prioritize resolution metrics

Authors

Community

Standardize artifact evaluation

Promote comprehensive benchmarking practices

18 / 44

Temporal Resolution

Time per sample and maximum sampling rate.

TunneLs for Bootlegging: Fully Reverse-Engineering GPU TLBs for Challenging Isolation Guarantees of NVIDIA MIG.

Z. Zhang, T. N. Allen, F. Yao, X. Gao, R. Ge. CCS'23.

Reviewers

Benchmarking Practices

Recommendations

Share artifacts

Prioritize resolution metrics

Authors

Community

Standardize artifact evaluation

Promote comprehensive benchmarking practices

18 / 44

Temporal Resolution

Time per sample and maximum sampling rate.

POWERT Channels: A Novel Class of Covert Communication Exploiting Power Management Vulnerabilities.

S. K. Khatamifard, L. Wang, A. Das, S. Köse, U. R. Karpuzcu. HPCA'19.

Reviewers

Benchmarking Practices

Recommendations

Share artifacts

Prioritize resolution metrics

Authors

Community

Standardize artifact evaluation

Promote comprehensive benchmarking practices

18 / 44

Reviewers

Demand relevant benchmarks

Focus on core properties

Benchmarking Practices

Recommendations

Share artifacts

Prioritize resolution metrics

Authors

Community

Standardize artifact evaluation

Promote comprehensive benchmarking practices

19 / 44

Share artifacts

Prioritize resolution metrics

Authors

Reviewers

Community

Demand relevant benchmarks

Focus on core properties

Standardize artifact evaluation

Promote comprehensive benchmarking practices

How should microarchitectural side-channel attacks be meaningfully evaluated and compared?

Benchmarking Practices

Takeaway

Practices (resolution metrics...)

Outdated Methodologies

Unfair Comparisons

Reproducibility Issues

Practices

Research Questions

How can developers be effectively encouraged to write constant time code?

Security Tools Adoption

How should microarchitectural side-channel attacks be meaningfully evaluated and compared?

Benchmarking Practices

How effective are modern defenses, and in particular fingerprinting techniques, against LLM Bots?

LLM Bots Detection

20 / 44

Semi-Automated and Easily Interpretable Side-Channel Analysis for Modern JavaScript.

I. Fayolle, J. Wichelmann, A. Köhl, W. Rudametkin, T. Eisenbarth, C. Maurice. CANS'24.

21 / 44

Control flow

Memory usage / allocation

Vulnerable Code

Control flow

Memory usage / allocation

Constant Time Code

Security Tools Adoption

Context

CacheAudit

Binsec/Rel

ctgrind

CacheQL

DiffFuzz

ct-fuzz

Microwalk

Vulnerability Analysis Tools

...

Stack Overflow developer survey 2025. Stack Overflow. https://survey.stackoverflow.co/2025/, accessed: 09-14-2026.

Python developers survey 2024 results. jetbrains. https://lp.jetbrains.com/python-developers-survey-2024/, accessed: 09-14-2026.

Not integrated into development processes

Compatibility problems

Not well documented

Poor usability

Problems

Security Tools Adoption

Context

21 / 44

They're not that hard to mitigate: What Cryptographic Library Developers Think About Timing Attacks.

J. Jancar, M. Fourné, D. De Almeida Braga, M. Sabt, P. Schwabe, G. Barthe, P.-A. Fouque, Y. Acar. S&P'22.

"These results must be false": A usability evaluation of constant-time analysis tools.

M. Fourné, D. De Almeida Braga, J. Jancar, M. Sabt, P. Schwabe, G. Barthe, P.-A. Fouque, Y. Acar. USENIX'24.

Analyses during Continuous Integration (CI)

Statistical + Dynamic Analysis

Quickly Locates and Quantifies Leakages

Microwalk-CI: Practical Side-Channel Analysis for JavaScript Applications.

J. Wichelmann, F. Sieck, A. Pätschke, T. Eisenbarth. CCS'22.

Microwalk: A Framework for Finding Side Channels in Binaries.

J. Wichelmann, A. Moghimi, T. Eisenbarth, B. Sunar. ACSAC'18.

22 / 44

Security Tools Adoption

Microwalk-CI: Presentation

Will their execution paths be different?

YES

YES

23 / 44

Security Tools Adoption

Microwalk-CI: Workflow

Modern        cannot be analyzed

Post-2015 ECMAScript Versions

Deferred Execution

ES Modules (ESM)

Classes

Asynchronous Programming

Jalangi2

Compatibility Issue

Limits

24 / 44

Security Tools Adoption

Microwalk-CI: Limits

Compatibility Issue

Limits

Tedious Test Writing

Writing tests is tedious and error-prone

Security Tools Adoption

Microwalk-CI: Limits

24 / 44

Compatibility Issue

Limits

Tedious Test Writing

Limited Report Interpretability

The final report is both impractical and incomplete

Security Tools Adoption

Microwalk-CI: Limits

24 / 44

Compatibility Issue

Limits

Tedious Test Writing

Limited Report Interpretability

Solutions

Actionable Final Report

Semi-Automated Test Writing Process

Modern JavaScript Support

Security Tools Adoption

Microwalk-CI: Limits

24 / 44

Excerpt from the aes-js analysis target.

Automated

Test Case Generation

Subtargets Introduction

25 / 44

Security Tools Adoption

Semi-Automated Test Writing Process

code-coverage

code-coverage

Leak Information Box

Detailed Header

Extract of the HTML report after analyzing the @noble/ciphers library.

26 / 44

Security Tools Adoption

Actionable Final Report

Vulnerabilities

Performance

Development Effort

Test on Modern

Old        vs. New

aes-js

node-forge

noble@ciphers  noble@curves

Maximize code-coverage 

Replicate Wichelmann CCS'22

Find more leakages

Identify missed code path

Support Modern

Perform better

135s vs. 73s ; < 2GB

80%

45%

93%

72%

Need 1 file per library

464 lines vs. 206 lines

27 / 44

Security Tools Adoption

Evaluation

Find more vulnerabilities with less effort

Compatibility Issue

Limits

Tedious Test Writing

Limited Report Interpretability

Solutions

Actionable Final Report

Semi-Automated Test Writing Process

Modern JavaScript Support

Wider adoption requires improved usability

How can developers be effectively encouraged to write constant time code?

28 / 44

Security Tools Adoption

Takeaway

Research Questions

How effective are modern defenses, and in particular fingerprinting techniques, against LLM Bots?

LLM Bots Detection

How should microarchitectural side-channel attacks be meaningfully evaluated and compared?

Benchmarking Practices

How can developers be effectively encouraged to write constant time code?

Security Tools Adoption

29 / 44

On the Internet, Nobody Knows You’re an LLM Bot: Unmasking Web Agents with Multi-Layer Fingerprinting.

I. Fayolle, S. Bouhenniche, S. Pélissier, P. Laperdrix, C. Maurice, W. Rudametkin. arXiv, 2026. (Under Submission)

Global Internet Traffic for the past 10 years.

Bad bot report: the rapid rise of bots and the unseen risk for business. Imperva. 2025.

30 / 44

LLM Bots Detection

Context

HTTP-Based Scrapers

Large-Scale Data Extraction

Execute JavaScript

cURL     wget    scrapy

LLM-Based Chatbot

Browser Automation

Framework (BAF)

Execute JavaScript

Selenium    Puppeteer Playwright

Scripted Interactions

Control Web Browser

Web Agents

Natural Language Instructions

BAF + LLM-Based Reasonning

Large-Scale High Value Data Extraction

OpenClaw     Browser Use   Skyvern   ChatGPT Agent ...

Integrate Stealth Mechanisms

31 / 44

LLM Bots Detection

Taxonomy of Bots

IP: 20.42.10.176

UA: GPTBOT

*************

*****

IP address reputation

Rule-based detection

32 / 44

LLM Bots Detection

Anti-Bot Mechanisms

IP address reputation

Rule-based detection

LLM Bots Detection

Anti-Bot Mechanisms

32 / 44

IP: 20.42.10.176

UA: Mozilla/5.0

 (iPhone14,2; *******)

*****

CAPTCHAs

IP address reputation

Rule-based detection

LLM Bots Detection

Anti-Bot Mechanisms

32 / 44

CAPTCHAs

Proof-of-work

IP address reputation

Rule-based detection

LLM Bots Detection

Anti-Bot Mechanisms

32 / 44

CAPTCHAs

Proof-of-work

Fingerprinting

Screen width: 1920;
Screen height: 1080;
Timezone: UTC+02:00;
 
 

IP: 20.42.10.176

UA: Mozilla/5.0

 (iPhone14,2; *******)

*****

IP address reputation

Rule-based detection

LLM Bots Detection

Anti-Bot Mechanisms

32 / 44

IP address reputation

Rule-based detection

CAPTCHAs

Proof-of-work

Fingerprinting

Proprietary Solutions

LLM Bots Detection

Anti-Bot Mechanisms

32 / 44

33 / 44

LLM Bots Detection

Overview

IP address reputation

Rule-based detection

CAPTCHAs

Proof-of-work

Fingerprinting

Proprietary Solutions

 Can discriminative attributes uniquely identify bots?

Scrapers

Web Agents

BAF

Do bots pass defenses?

Data collection infrastructure for the deployment and instrumentation of honeysites.

LLM Bots Detection

Experimental Setup

34 / 44

Data collection infrastructure for the deployment and instrumentation of honeysites.

LLM Bots Detection

Experimental Setup

34 / 44

IP address source port

IP address  User-Agent

cookie_id

DateTime

IP address cookie_id User-Agent DateTime

source port

 

 

Data collection infrastructure for the deployment and instrumentation of honeysites.

IP address User-Agent

cookie_id

DateTime

IP address cookie_id User-Agent DateTime

source port

 

 

LLM Bots Detection

Experimental Setup

34 / 44

IP address source port 

IP address  User-Agent

cookie_id

DateTime

IP address cookie_id User-Agent DateTime

source port

 

 

Data collection infrastructure for the deployment and instrumentation of honeysites.

LLM Bots Detection

Experimental Setup

34 / 44

Subjects

Selenium, Puppeteer, Playwright

Local vs. Cloud

Open- vs. Close-Source

Evasion Capabilities

Characteristics

LLM Bots Detection

Methodology

OpenClaw,

ChatGPT Agent,

Skyvern,

Browser Use,

Crawl4AI,

Claude for Chrome

Chrome v144.0.7559.96, Firefox v136.0, and v147.0.1

cURL, wget, scrapy

35 / 44

Subjects

Honeysite appearance: baseline site.

Prompt.

LLM Bots Detection

Methodology

35 / 44

Subjects

Anti-Bot Mechanism

Timeout

Blocked

Passed

Honeysite appearance: baseline site.

LLM Bots Detection

Methodology

35 / 44

IP

TLS

Browser Fingerprint

Prompt output

Subjects

Defense passed?

Visit

1,449 Initiated Visits

   1,383 Visits with fingerprints correctly linked

 

From 01-28 to 02-24-2026

LLM Bots Detection

Dataset

36 / 44

Bot access to honeysites under different anti-bot mechanisms.

Takeaway

JavaScript-based defense are effective against    

LLM Bots Detection

Evaluating Anti-Bot Mechanisms

37 / 44

Takeaway

JavaScript-based defense are effective against    

            struggles with interactive challenges

Bot access to honeysites under different anti-bot mechanisms.

LLM Bots Detection

Evaluating Anti-Bot Mechanisms

37 / 44

Takeaway

JavaScript-based defense are effective against    

            struggles with interactive challenges

Some         bypass all defense

Bot access to honeysites under different anti-bot mechanisms.

LLM Bots Detection

Evaluating Anti-Bot Mechanisms

37 / 44

Takeaway

JavaScript-based defense are effective against    

            struggles with interactive challenges

Some         bypass all defense

Stealth mode may increase detectability

Bot access to honeysites under different anti-bot mechanisms.

LLM Bots Detection

Evaluating Anti-Bot Mechanisms

37 / 44

IP

TLS

Browser Fingerprint

LLM Bots Detection

Characterizing Bots

Takeaway

Fixed execution environments

       stable fingerprints

Stealth mechanisms

    inconsistencies

Cloud-Based bots &         easier to detect

Local bots inherit the user's trusted IP address

Cloud          can connect through:

Datacenter infrastructure ;

Residential proxies

Discriminative Attributes

User-Agent ; Screen Resolution/Display ; Permissions State ; Cookie Management ; Number of CPU Cores ; Stealth Behavior ; Tool-Specific Attributes

IP

TLS Fingerprint

38 / 44

LLM Bots Detection

Multi-Layer Bots Classification

Classification Evaluation using Random Forest

(with alea ~7%).

 Can discriminative attributes uniquely identify bots?  14 classes (Human, Chatgpt Agent, Browser Use Local, ...)

Discriminative Attributes

User-Agent ; Screen Resolution/Display ; Permissions State ; Cookie Management ; Number of CPU Cores ; Stealth Behavior ; Tool-Specific Attributes

IP

TLS Fingerprint

39 / 44

Takeaway

Individual fingerprinting =

Combined fingerprinting =

Stealth Mode

IP 

TLS

Browser

...

...

...

Hard Challenges

LLM Bots Detection

Takeaway

How effective are modern defenses, and in particular fingerprinting techniques, against LLM Bots?

40 / 44

Conclusion

Side Channels in Web Environments

From Microarchitectural Attacks to Developer-Centric Countermeasures

IP 

TLS

Browser

Hard Challenges

...

...

Find more vulnerabilities with less effort

Vulnerable Code

Low level

Microarchitecture

High level

Web-based applications & systems

41 / 44

Practices

Recommendations

Conclusion

IP 

TLS

Browser

Hard Challenges

...

...

Find more vulnerabilities with less effort

Vulnerable Code

Low level

Microarchitecture

High level

Web-based applications & systems

New Benchmark

Extend Microwalk

New defense

What's next?

42 / 44

Side Channels in Web Environments

From Microarchitectural Attacks to Developer-Centric Countermeasures

Practices

Recommendations

Conclusion

The Challenges of Bringing Cryptography from Research Papers to Products: Results from an Interview Study with Experts.

 

K. Fischer, I. Trummová, P. Gajland, Y. Acar, S. Fahl, M. A.Sasse. USENIX'24.

LLMs Cannot Reliably Identify and Reason About Security Vulnerabilities (Yet?): A Comprehensive Evaluation, Framework,

and Benchmarks. S. Ullah, M. Han, S. Pujar, H. Pearce, A. K. Coskun, G. Stringhini. S&P'24.

In generative AI we trust: can chatbots effectively verify political information?

E. Kuznetsova, M. Makhortykh, V. Vziatysheva, M. Stolze, A. Baghumyan, A. Urman. Journal of Computational Social Science, 2025.

What News Do People Get on Social Media? Analyzing Exposure and Consumption of News through Data Donations.

S. Chouaki, A. Chakraborty, O. Goga, S. Zannettou, WWW'24.

43 / 44

How should microarchitectural side-channel attacks be meaningfully evaluated and compared?

How can developers be effectively encouraged to write constant time code?

How effective are modern defenses, and in particular fingerprinting techniques, against LLM Bots?

Summary

Benchmarking Practices

Security Tools Adoption

LLM Bots Detection

On the Internet, Nobody Knows You’re an LLM Bot: Unmasking Web Agents with Multi-Layer Fingerprinting.

I. Fayolle, S. Bouhenniche, S. Pélissier, P. Laperdrix, C. Maurice, W. Rudametkin. arXiv, 2026. (Under Submission)

Semi-Automated and Easily Interpretable Side-Channel Analysis for Modern JavaScript.

I. Fayolle, J. Wichelmann, A. Köhl, W. Rudametkin, T. Eisenbarth, C. Maurice. CANS'24.

Practice Makes (Im)Perfect: A Look Back at Benchmarking Practices for Microarchitectural Side-Channel Attacks.

I. Fayolle, A. Geimer, D. De Almeida Braga, C. Maurice. CCS'26.

44 / 44

Flaws and their impact on evaluation pro.

Backup

Benchmarking Practices

Backup

Benchmarking Practices

Summary of benchmarking flaws across all evaluated papers.

Backup

Benchmarking Practices

Prevalence of the 19 flaws in the 83 surveyed papers

Fig. 4: Distribution of reported properties in covert-channel evaluations.

Fig. 5: Distribution of side-channel attack types.

59 (Speed)

30 (Speed, Error Rate)

Reported Metrics per Paper

18 (Speed, Error Rate, ...)

218 Attacks

71 Covert Channels

147 Side Channels

Attacked Platforms per Paper

73 CPUs (and 65 Intel)

9 GPUs (and 3 iGPUs)

9 TEEs

Backup

Benchmarking Practices

Security

Architecture

>> Flaws

Number of flaws per paper over time.

Security

Architecture

Backup

Benchmarking Practices

Benchmarked platforms per paper over time across all conferences.

>> Evaluated Platforms

Security

Architecture

>> Flaws

Backup

Benchmarking Practices

Number of evaluations per paper over time across all conferences.

Backup

Benchmarking Practices

Reported properties in covert-channel evaluations across conference.

Backup

Benchmarking Practices

    Ensure fair comparisons

     Define & report errors clearly

     Test cross-platform generalization

   Be explicit about evaluation details

   Evaluate every claim empirically

   Share artifacts

     Prioritize resolution metrics

Authors

Backup

Benchmarking Practices

Authors

Avoid requiring full crypto attacks

Push for realistic setups

   Demand relevant benchmarks

     Focus on core properties

Reviewers

Backup

Benchmarking Practices

Authors

Encourage meaningful metrics

Foster robust benchmarking culture

Standardize artifact evaluation

Promote complete reporting

Community

Reviewers

Backup

Benchmarking Practices

Trace generation pipeline. The red arrows denote execution flow.

Modern

Backwards-Compatible JavaScript

         Instrumentation

Backup

Security Tools Adoption

Comparison between Microwalk V3.2.0 and Microwalk using the enhanced analysis

toolchain on aes-js and node-forge libraries.

Backup

Security Tools Adoption

Results of the enhanced analysis toolchain with 2 modern JavaScript

cryptographic libraries: @noble/ciphers ; @noble/curves.

Backup

Security Tools Adoption

Feature comparison of selected Web Agents.

Backup

LLM Bots Detection

Normal Prompt (i) and Special Prompt (ii).

Backup

LLM Bots Detection

1,449 Initiated Visits

   1,383 Visits with fingerprints correctly linked

 

From 01-28 to 02-24-2026

Collected visits per tool.

Backup

LLM Bots Detection

Bot access to honeysites under different anti-bot mechanisms.

Takeaway

JavaScript-based defense are effective against    

            struggles with interactive challenges

Some         bypass all defense

Stealth mode may increase detectability

Backup

LLM Bots Detection

IP

Takeaway

Local tools inherit the user's trusted IP address

Cloud          can connect through:

1. Datacenter infrastructure

2.  Residential proxies

IP-layer characteristics per tool.

Backup

LLM Bots Detection

Backup

LLM Bots Detection

TLS Client Hello Packet obtained with Wireshark

Mapping between JA4 index, full JA4 and its features.

Backup

LLM Bots Detection

IP

TLS

 JA4 Variability

With pre_shared_key (PSK) or not

QUIC vs. TCP

Takeaway

Cloud-Based bots &        are easier to detect

Stealth mode has no effect on JA4

Dominant JA4 and discriminative power per tool.

Backup

LLM Bots Detection

A
t, t'
v

: Attribute

: Tools

: Value

A-Score

A\text{-}Score(A, t) = \sum_{v \in Vals(A, t)} V\text{-}Score(A, v, t)
V\text{-}Score(A, v, t) = Intra(A, v, t) \times Inter(A, v, t)

V-Score

Inter(A, v, t) = 1 - P(A = v \mid t' \neq t)

Inter-tool Value Exclusivity

Intra-tool Value Probability

Intra(A, v, t) = P(A = v \mid t)

How stable and unique an attribute value is for a tool ?

How stable and unique an attribute  is for a tool ?

∑
v

Backup

LLM Bots Detection

Discriminative Attributes

User-Agent

ChatGPT Agent:

Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/141.0.0.0 Safari/537.36

Skyvern:

Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/144.0.0.0 Safari/537.36 Edg/144.0.0.0

IP

TLS

Browser Fingerprint

Backup

LLM Bots Detection

Discriminative Attributes

User-Agent

Screen Resolution and Display

ChatGPT Agent:

1280 × 960

Crawl4AI:

1080 × 600

IP

TLS

Browser Fingerprint

Backup

LLM Bots Detection

Discriminative Attributes

User-Agent

Screen Resolution and Display

Permissions State

ChatGPT Agent & Crawl4AI:

Not Supported

BrowserUse:

Denied

Skyvern & Claude:

Prompt or Granted

IP

TLS

Browser Fingerprint

Backup

LLM Bots Detection

Discriminative Attributes

User-Agent

Screen Resolution and Display

Permissions State

Cookie Management

Claude:

Share Cookie with Human

ChatGPT Agent, Skyvern & OpenClaw:

Reused Cookie

BrowserUse:

New Cookie

IP

TLS

Browser Fingerprint

Backup

LLM Bots Detection

Discriminative Attributes

User-Agent

Screen Resolution and Display

Permissions State

Cookie Management

Number of CPU Cores

Skyvern:

32

ChatGPT Agent:

13

BrowserUse:

From 2 to 64

IP

TLS

Browser Fingerprint

Backup

LLM Bots Detection

Discriminative Attributes

User-Agent

Screen Resolution and Display

Permissions State

Cookie Management

Number of CPU Cores

Stealth Behavior

Crawl4AI Stealth:

Modify User-Agent

Inject synthetic referer

IP

TLS

Browser Fingerprint

Backup

LLM Bots Detection

IP

TLS

Browser Fingerprint

Discriminative Attributes

User-Agent

Screen Resolution and Display

Permissions State

Cookie Management

Number of CPU Cores

Tool-Specific Attributes

Stealth Behavior

ChatGPT Agent:

signature & signature-agent

Crawl4AI:

"default" for the notification permission

Backup

LLM Bots Detection

Setup

Random Forest, XGBoost, CatBoostClassifier

1370 Fingerprints (      )

80% Train (SMOTE) / 20% Test

Run 10 times with random seeds

Classification Evaluation using Random Forest.

Backup

LLM Bots Detection

Confusion matrices for individual fingerprinting layers with Random Forest.

Backup

LLM Bots Detection

Confusion matrices for combined fingerprinting layers with Random Forest.

Backup

LLM Bots Detection

PhD Thesis Defense

By Iliana Fayolle

PhD Thesis Defense

  • 20