AI vs. AI

Real Attacking and Defending in Cybersecurity

Sebastian Garcia. Stratosphere Lab. CTU University

Stratosphere lab

AI and ML for Cybersecurity to help others.

https://www.stratosphereips.org/

  • At the beginning
    • Simpler attacks
    • Simpler defenses

the cybersecurity problem

  • Governments and companies got deeper
    • Zero-day markets
    • LoL techniques
    • Targetted victims
    • More complex and larger attack surface
  • Are attackers using AI for the attack?
    • Not really
    • Unclear for malware
      • Many "reports", no evidence
    • For phishing and propaganda creation. Yes
    • No need for AI in malware and nets. Evasion is easy
  • AI is not very autonomous. Yet

The ai problem?

In the Stratosphere Lab

  • Better AI malware for AV evasion [pdf]
  • AI Evasion of Network IDS [pdf]
  • XAI [pdf]
  • Honeypots with AI [pdf]
  • Attackers with AI [pdf]

Ai for cybersecurity

In the Stratosphere Lab

  • Network IDS with AI [code]
    • Behavioral ML
    • Flow-based ML
    • DNS anomalies
    • DGA detection with ML
    • Federated Learning
    • P2P TI sharing

Ai for cybersecurity

Aidojo

  • Multiagent
  • IPs change
  • Actions and parameters
  • Scenarios
  • State space: 4 × 10^18
    • Chess is 10^43
  • Goals
  • Randomness
  • Rewards
  • When to win?

aidojo - the environment

  • Q-learning
  • Sarsa
  • LLM
  • GA+Markov Chains

aidojo - attacking agents

  1. QAgent Action selected ScanNetwork|{'target_network': 192.168.1.0/24, 'source_host': 192.168.2.6}>
  2. QAgent Action selected FindServices|{'target_host': 192.168.1.2, 'source_host': 192.168.2.6}>
  3. QAgent Action selected ExploitService|{'target_host': 192.168.1.2, 'target_service': Service(name='microsoft-ds', type='passive', version='10.0.19041', is_local=False), 'source_host': 192.168.2.6}>
  4. QAgent Action selected FindData|{'target_host': 192.168.1.2, 'source_host': 192.168.1.2}>
  5. QAgent Action selected ExfiltrateData|{'target_host': 213.47.23.195, 'source_host': 192.168.1.2, 'data': Data(owner='User1', id='DataFromServer1', size=0, type='')}>

    QAgent Summary: Steps=5. Reward 100. States in Q_table = 1454300

aidojo - qlearning attacking

aidojo - LLM attacking

aidojo - LLM attacking

aidojo - LLM attacking

aidojo - llm attacking

  • Models
    • GPT-4-turbo-preview
    • GPT-3.5-turbo
    • Fine-tuned 7B models based on Zephyr
    • GPT-4 is as ‘good’ as humans
    • Local models are better than GPT-3.5
  • ​When they win…

    • They generalize to any environment

    • They do not need further training

aidojo - defending

  • Algorithms
    • Q-learning
    • Random
  • New needs
    • The real problem: Benign traffic for FP
    • What does it mean to win?
  • Be realistic
    • Need to copy logs from hosts
    • Analyze and decide what to block

shellm - llm for defense

This computer does not exists

shellm - llm for defense

  • An LLM-based SSH honeypot
  • All you see is just text, nothing is executed
  • Dynamic content while you type
  • No preconfigured content
  • Content and behaviour are realistic, and they adapts
  • Part of the Velmes Suite of more LLM honeypots
    • POP3
    • SMTP
    • HTTP
    • MYSQL

https://arxiv.org/abs/2309.00155, https://github.com/stratosphereips/shelLM​

https://github.com/stratosphereips/VelLMes-AI-Deception-Framework

shellm - llm for defense

Can we demo?

shellm - llm for defense

Personality Prompt

  • Instructs how to behave and respond

  • Needs to be carefully and iteratively developed

  • Includes the main “personality”

    • “You are a Linux shell. Respond only to Linux commands.”

  • But also many examples of desired behavior

  • Many instructions to avoid pitfalls

  • Also, fine-tunning

shellm - llm for defense

  • Fine-tuning

    • Training of the LLM for specific tasks

    • A much smaller dataset is needed

    • Our dataset had 112 training and 21 validation samples

  • After fine-tuning, the personality prompt is much shorter

shellm - llm for defense

  • LLMs can be used as honeypots

  • 34 human attackers took part in the experiment

  • shelLM outperformed Cowrie

  • Fooled 1/2 of the attackers

Evaluation of deception capabilities

shellm - llm for defense

ssh tomas@olympus.felk.cvut.cz -p 1337

Password: tomy

Want to play yourself?

ARACNE - llm for attack

An autonomous LLM Attacker for on real SSH servers

  • Can we use LLM for attacking? Yes

  • Not attacking only, but planning, reasoning, and executing

  • The user just gives a goal:

    • ​"Encrypt all the files and leave a message in the home folder to pay in this email myeamil@whatever.com"

https://www.stratosphereips.org/blog/2025/2/24/introducing-aracne-a-new-llm-based-shell-pentesting-agent

ARACNE - llm for attack

ARACNE - llm for attack

Demo!

ARACNE vs SHellm

ARACNE vs Bandit

https://overthewire.org/wargames/bandit/

1440 experiments

Agentic Architectures

  • Not everything works the same

    • For us:

      • Interpreter llama3.1

      • Planner o3-mini-2025-01-31

      • Summarizer gpt-4o-2024-08-06

  • Most LLMs have huge variability

  • Jail breaking

  • Memory is crucial

  • SFT can help a lot. Probably mandatory

Introduction to security mooc

https://cybersecurity.bsy.fel.cvut.cz/

Thanks a lot!

Sebastian Garcia

 

https://bsky.app/profile/eldraco.bsky.social

https://infosec.exchange/@eldraco
https://www.linkedin.com/in/sebagarcia/
http://stratosphereips.org