AI vs. AI

Real Attacking and Defending in Cybersecurity

Sebastian Garcia. Stratosphere Lab. CTU University

https://www.stratosphereips.org/

the cybersecurity problem

• Governments and companies got deeper
  • Zero-day markets
  • LoL techniques
  • Targetted victims
  • More complex and larger attack surface

• Are attackers using AI for the attack?
  • Not really
  • Unclear for malware
    • Many "reports", no evidence
  • For phishing and propaganda creation. Yes
  • No need for AI in malware and nets. Evasion is easy
• AI is not very autonomous. Yet

The ai problem?

In the Stratosphere Lab

• Better AI malware for AV evasion [pdf]
• AI Evasion of Network IDS [pdf]
• XAI [pdf]
• Honeypots with AI [pdf]
• Attackers with AI [pdf]

Ai for cybersecurity

In the Stratosphere Lab

• Network IDS with AI [code]
  • Behavioral ML
  • Flow-based ML
  • DNS anomalies
  • DGA detection with ML
  • Federated Learning
  • P2P TI sharing

Ai for cybersecurity

Aidojo

• Multiagent
• IPs change
• Actions and parameters
• Scenarios
• State space: 4 × 10^18
  • Chess is 10^43
• Goals
• Randomness
• Rewards
• When to win?

aidojo - the environment

• Q-learning
• Sarsa
• LLM
• GA+Markov Chains

aidojo - attacking agents

1. QAgent Action selected ScanNetwork|{'target_network': 192.168.1.0/24, 'source_host': 192.168.2.6}>
2. QAgent Action selected FindServices|{'target_host': 192.168.1.2, 'source_host': 192.168.2.6}>
3. QAgent Action selected ExploitService|{'target_host': 192.168.1.2, 'target_service': Service(name='microsoft-ds', type='passive', version='10.0.19041', is_local=False), 'source_host': 192.168.2.6}>
4. QAgent Action selected FindData|{'target_host': 192.168.1.2, 'source_host': 192.168.1.2}>
5. QAgent Action selected ExfiltrateData|{'target_host': 213.47.23.195, 'source_host': 192.168.1.2, 'data': Data(owner='User1', id='DataFromServer1', size=0, type='')}>
   
   QAgent Summary: Steps=5. Reward 100. States in Q_table = 1454300

aidojo - qlearning attacking

aidojo - LLM attacking

aidojo - LLM attacking

aidojo - LLM attacking

aidojo - llm attacking

• Models
  • GPT-4-turbo-preview
  • GPT-3.5-turbo
  • Fine-tuned 7B models based on Zephyr
  • GPT-4 is as ‘good’ as humans
  • Local models are better than GPT-3.5
• ​When they win…

  • They generalize to any environment

  • They do not need further training

aidojo - defending

• Algorithms
  • Q-learning
  • Random
• New needs
  • The real problem: Benign traffic for FP
  • What does it mean to win?
• Be realistic
  • Need to copy logs from hosts
  • Analyze and decide what to block

shellm - llm for defense

This computer does not exists

shellm - llm for defense

• An LLM-based SSH honeypot
• All you see is just text, nothing is executed
• Dynamic content while you type
• No preconfigured content
• Content and behaviour are realistic, and they adapts
• Part of the Velmes Suite of more LLM honeypots
  • POP3
  • SMTP
  • HTTP
  • MYSQL

https://arxiv.org/abs/2309.00155, https://github.com/stratosphereips/shelLM​

https://github.com/stratosphereips/VelLMes-AI-Deception-Framework

shellm - llm for defense

Can we demo?

shellm - llm for defense

Personality Prompt

• Instructs how to behave and respond

• Needs to be carefully and iteratively developed

• Includes the main “personality”

  • “You are a Linux shell. Respond only to Linux commands.”

• But also many examples of desired behavior

• Many instructions to avoid pitfalls

• Also, fine-tunning

shellm - llm for defense

• Fine-tuning

  • Training of the LLM for specific tasks

  • A much smaller dataset is needed

  • Our dataset had 112 training and 21 validation samples

• After fine-tuning, the personality prompt is much shorter

shellm - llm for defense

• LLMs can be used as honeypots

• 34 human attackers took part in the experiment

• shelLM outperformed Cowrie

• Fooled 1/2 of the attackers

Evaluation of deception capabilities

shellm - llm for defense

ssh tomas@olympus.felk.cvut.cz -p 1337

Password: tomy

Want to play yourself?

ARACNE - llm for attack

An autonomous LLM Attacker for on real SSH servers

• Can we use LLM for attacking? Yes

• Not attacking only, but planning, reasoning, and executing

• The user just gives a goal:

  • ​"Encrypt all the files and leave a message in the home folder to pay in this email myeamil@whatever.com"

https://www.stratosphereips.org/blog/2025/2/24/introducing-aracne-a-new-llm-based-shell-pentesting-agent

ARACNE - llm for attack

ARACNE - llm for attack

Demo!

ARACNE vs SHellm

ARACNE vs Bandit

https://overthewire.org/wargames/bandit/

1440 experiments

Agentic Architectures

• Not everything works the same

  • For us:

    • Interpreter llama3.1

    • Planner o3-mini-2025-01-31

    • Summarizer gpt-4o-2024-08-06

• Most LLMs have huge variability

• Jail breaking

• Memory is crucial

• SFT can help a lot. Probably mandatory

Introduction to security mooc

https://cybersecurity.bsy.fel.cvut.cz/

Thanks a lot!

Sebastian Garcia

https://bsky.app/profile/eldraco.bsky.social

https://infosec.exchange/@eldraco
https://www.linkedin.com/in/sebagarcia/
http://stratosphereips.org