How to backdoor the Linux Kernel

(and fail?)

Konstantin Ryabitsev

Linux Security Summit

Vancouver, 2023

  • I live in Montréal, Québec

  • Linux admin since 1997

  • (formerly) head of LF Infrastructure Security

    • now just head of Core IT Projects team

  • in charge of for the past 10+ years

    • keeper of grounds and keys

About me

Did anyone ever, you know, approach you with some shady offer?

– Common Question


Price $149 /mo

One zero-day

Two backdoors

7-day turnaround

Plausible deniability


Price $299 /mo


Three zero-days

Three backdoors

7-day turnaround

Plausible deniability


Price $599 /mo

Up to five zero-days

Five backdoors

3-day turnaround

Plausible deniability

No, they haven't

On parle français, se habla español, 厕所在哪里

I'm kidding

Please don't contact me. It will be a very short awkward conversation and then I'll have to figure out how to contact both the FBI and the CSIS.

  • A hidden vulnerability

  • Installed by the victim

    • i.e. not added by attacker after a compromise

  • Allowing remote access

  • Or elevating privileges

  • Or exfiltrating sensitive info

    • encryption keys

    • or secret docs

WHat do I mean by "backdoor"

This is not "How to Backdoor Linux for dummies."
Check if you're in the right talk.

Attacking the Pipeline


  • developer submits patches
  • maintainer reviews patches
  • maintainer accepts patches


  • Maintainer sends git pull
  • Linus reviews git pull
  • Linus merges git pull


  • Linus tags a release
  • Greg KH signs a tarball
  • user downloads tarball
  • Can you sneak malicious code inside a patch?
  • Can you hack/threaten/bribe LF admins to add malicious code to a patch?
  • Can you trick the maintainer to apply a malicious patch?

Submission stage

  • Can you send a malicious pull request to Linus?
  • Can you hack Linus and modify his repo?
  • Can you make Linus sign a malicious tag?

Merge stage

  • Can you make Greg KH sign the wrong tarball?
  • Can you hack/threaten/bribe LF admins to replace a tarball with a malicious one?
  • Can you MiTM a download from

Publish stage

  • Developer submits code
  • Maintainer reviews code
  • Maintainer accepts code
  • Maintainer sends a pull request

Attacking the patch workflow

What you think the workflow is like

What it's actually like

  • Unlikely to succeed for complex backdoors
    • maybe a simple vulnerability to elevate privs
    • not complex code to sniff and exfiltrate secrets
  • Hiding complex backdoors inside huge patchsets?
    • huge patchsets are rare for critical code paths
    • there are many eyes reviewing those patches
  • backdooring a relatively obscure device driver?
    • just volunteer to be a maintainer for that driver

Overtly sending malicious code

  • Probably not the best poster case
    • they were from “James Bond”
    • they were not trying very hard
    • the experience left us stirred, not shaken

UMN and "Hypocrite Commits"

  • Patches reviewed vs. patches applied
    • patches retrieved from lore or patchwork
      • "I remember reviewing that series"
      • "It's from Alex, whom I trust"
      • "DKIM signatures check out"
    • "here is a v12 with minor wording changes"

Covertly sneaking in malicious code

  • b4 verifications
    • DKIM signatures
      • fragile on many lists
      • requires trust in domain admins
      • typosquattable ( vs
    • Patatt signatures
      • DKIM-like, but end-to-end
      • Tailored to patches, resistant to ML junk
      • Can use PGP, SSH, or ED25519 keys
      • Require keyring management

End-to-end trust

  • b4 shazam
    • can turn a patch series into a pull request
    • shazam first, then review
  • b4 diff
    • can show you a diff between series
  • b4 send
    • simplifies series management
    • auto-signs before sending
  • b4 kr
    • keyring management (forthcoming)

New(er) features in b4

  • Likely will still be found out
    • there are eyes watching commits
    • (but they may not be talking)
  • Intentional bugs can still be caught by CI/bots
    • fuzzers
    • integration bots

If a malicious series got applied

  • Likely will be almost immediately discovered
  • There are tricks to make this more successful
    • arrange for their laptop to be stolen first
    • or their disk suddenly corrupted
    • force them to do a fresh full clone of the repo
  • Signing git commits helps
    • Yes, I know this is super annoying
    • it's an effective way to quickly verify the repository after a full re-clone
    • or when using shared repositories

Can a LF admin backdoor a repo?

  • Publishes information about each commit
  • Replicates to multiple mirrors
  • Tamper-evident (NOT tamper-resistant)
  • Can be used to exonerate developers
    • and point fingers at admins transparency log

  • has been rooted before
    • the worst has already happened
    • no guarantee it won't happen again
  • the #1 rule of is "do not trust"
    • we've always promoted zero-trust workflows
    • we promote end-to-end cryptography
    • we receive bits, we store bits, we send bits
      • it is your responsibility to verify them
      • we wrote tools to help with that

Trusting devs, not infra

  • Linus receives a pull request
  • Linus merges a pull request
  • Linus tags a release

Attacking Linus

  • All pull requests must be PGP-signed
    • assures the integrity of the entire git history
      • assuming we trust sha1
        • (which we still do)
          • ("ish")
  • Linus does verify signatures on all pull requests
  • It is very hard to get into Linus's keyring

This has been protected for a while

  • Yes, but with limited success
    • They can only backdoor their own patches
    • The original author will likely notice changes
      • they are very protective of their code
      • this may break CI in suspicious ways
  • If found out, their reputation will be destroyed
    • you probably want a key subsystem maintainer
    • so, this would be super expensive
    • there are much cheaper ways

Can you bribe a maintainer?

  • Yes, this would be fairly effective
  • There is a high chance it'll get caught
    • but it may not, until it's too late
    • There are people watching all commits
      • but they may not be talking to us
  • Still only effective for planting "intentional bugs"
    • large changes would be too noticeable

Can you hack their workstation?

  • Protect your digital identity above all
    • This usually means your crytographic keys
  • Keep your PGP keys on smartcards
    • you can get one for free if you're a maintainer
    • or get your DAYJOB to buy you one
  • Try to isolate your work env from your play env

Protect your workstations!

  • Greg KH tags a release
  • publishes a tarball
  • User (or distro) downloads the tarball

Attacking Downloads

  • All tarballs are signed with Greg's key
    • sigs are actually git notes in the stable repo
    • LF infra has no access to Greg's key
    • verifies signature before release is allowed to happen
  • The few times we messed up and broke the process, we were told about it very quickly
  • You are responsible for checking the signature of the tarball you downloaded

Also been protected for a while

  • We publish a tool that downloads random tarballs from mirrors and checks their signatures
  • The script runs from several undisclosed locations
  • You can help run it, too!

The sig-prover tool

You don't want to backdoor the kernel.

  • For every powerful state-level actor, there is another equally or more powerful state-level actor who is paying someone to watch all Linux commits

    • It’s like building a shared arsenal of powerful weapons that anyone can launch

  • For every powerful criminal syndicate there is another powerful criminal syndicate

    • They all keep money in the same Swiss banks

It won't be just *your* backdoor

If you wait long enough, someone will backdoor the kernel for you.

– Kees Cook [citation needed]

KSPP Report

Just wait until a critical vulnerability is fixed — it's not like manufacturers patch their kernels or anything.

– Greg KH [citation needed]

Are there backdoors in the Linux Kernel?

(c) user ebygomm on Flickr

  • Despite the perception of "bazaar", the Linux development model is actually quite rigorous
  • There is transparency and oversight at many levels
    • pre-commit and post-commit
  • Being a maintainer is a well-paying job with long-term job security
    • bribing a maintainer is super expensive
    • there are cheaper ways to root someone
  • Having root on infrastructure gets you almost nothing in terms of attacking the Linux source code

Intentional? Probably not.

  • The kernel you are using right now probably has a bad enough vulnerability in it to lead to local privilege escalation
    • just because it's not known to us, doesn't mean it's not known to the "bad guys"
    • and bad guys, what if it's known to your worst enemies and will be used against you?
  • Reporting critical bugs in the Linux kernel is in everyone's interest.

Unintentional? Probably yes.


How to backdoor the Linux Kernel

By Konstantin Ryabitsev

How to backdoor the Linux Kernel

Presentation to the Linux Security Summit, 2023

  • 244