The death of

the code review

When code costs nothing, how do you review it all?

Hello

What are we talking about today

Coding 8x faster to ship 1.3x faster

is a gigantic waste of effort

"The binding constraint appears to be shifting from writing code to reviewing, integrating, and ultimately distributing it."

- Leon Musolff, 8 Sept 2026

AI pull requests wait 4.6x longer for a reviewer, then get reviewed faster

The cost of producing plausible code has collapsed.

 

The cost of knowing whether to trust it has not.

Generation is no longer the bottleneck

  • Anthropic: 80% of merged production code is written by Claude
  • Google: 75% of new code is AI-generated
  • GitHub: monthly commits went from 1.4 billion in April to 2.9 billion in August

You can't just review harder

  • Cisco / SmartBear: 2,500 reviews, 3.2 million lines of code
  • Defect detection falls off past 200 to 400 lines in one sitting
  • Accuracy collapses past 450 lines an hour

This is what happens when you try

Some people decided

to stop reading the code

"Traditional human code review is no longer a necessary component of a software quality pipeline."

- Martin Monperrus, June 2026

"Code review is overrated, and the least valuable part of what humans add to software engineering."

- Charity Majors, August 2026

"Eyeballing every line of code has never been the most effective way to validate a change."

- Simon Willison, 22 Aug 2026

"We ditched code review at Duckbill (mostly)."

  • 5 engineers. 60 open pull requests. Two days of nothing but review.
  • Humans still review: public API, auth, design system, schema changes, agent skills
  • Pull requests merged per week: 80 to 154

"Humans may review pull requests, but aren't required to."

- OpenAI

METR: Passing tests

is not the same as mergeable

FrontierCode: A benchmark for

"would you merge this?"

80% pass if you ask the tests,

54% if you ask the maintainer.

Nobody has a grader they trust

  • OpenAI, 8 July: about 30% of SWE-Bench Pro's 731 public tasks are broken. Recommendation withdrawn.
  • FrontierCode 1.1, July: cheating detector added. Runs that look up the original PR score zero.

Everyone plateaus at about half

Anything cheap to verify gets beaten

Whoever writes the verifier writes next year's models

  • Cognition wrote FrontierCode. On 10 Sept it shipped SWE-2, trained to score on it: 50.0%, "within a point of Fable 5.1 at 64% lower cost."
  • Cursor's Bugbot turns your team's accept and reject clicks into "learned rules": 44,000 rules across 110,000 repos.

"If the same AI that wrote the code also reviews it, that doesn't really solve the problem. The review just isn't of the same quality."

- Leon Musolff, 8 Sept 2026

Who's actually running automated review right now?

  • GitHub Copilot: 60 million reviews, more than 1 in 5 code reviews on GitHub
  • 12,000 organizations run it on every pull request
  • 71% of reviews surface something actionable. 29% say nothing at all.

On 1 September, Copilot's review started counting as an approval

  • "Copilot can submit an approval that counts toward the repository's required-approvals rule."
  • Off by default. Admins can scope it to specific file paths.

The whole job is managing false positives

"Did the human accept it"

is the rule that works

How good are they, measured?

  • c-CRAB (National University of Singapore): Devin, Claude Code, Codex and PR-agent together solve about 40% of review tasks
  • SWE-PRBench: frontier models find 15% to 31% of the issues human reviewers flagged
  • Martian Code Review Bench: the best tool finds 63% of known issues

The machines inherit the Cisco curve

Four reviewers. Zero agreement.

Can you get away

without any human review?

A million lines of Rust in eleven days

  • 64 Claude agents. 6,502 commits. +1,009,272 lines. $165,000 of tokens.
  • Reviewer agents were given only the diff and "told to assume the code is wrong."
  • 11 rounds of security review. 24/7 fuzzing. 1.3 million test assertions. Zero tests skipped.

13,365 assertions the tests never looked at

What skipping review actually costs

"Please, please read the code."

Amazon now requires a senior engineer to sign off

on AI-assisted changes

  • Internal memo, March 2026: a "trend of incidents" with "high blast radius" from "Gen-AI assisted changes"
  • Junior and mid-level engineers can no longer ship AI-assisted code to production without senior sign-off

"The changes made it past multiple human and automated code reviews, as well as unit tests, end-to-end tests, automated verification, and dogfooding."

- Anthropic, 23 April 2026

Engineers now review the review,

not the code

  • "Engineers typically review the review itself, and not usually the code." - Gergely Orosz, 8 Sept 2026
  • Weaviate: 90% left to agents. Humans decide scope and exit criteria.
  • Anthropic and OpenAI: triage by blast radius.

"Review is the control point through which a coding agent's effect on software is decided. AI does not fix the sign of that effect: the team sets it."

- Carnegie Mellon, July 2026

The human moves up a level

Someone still has to

review the reviewer

  • "This action is not hardened against prompt injection attacks and should only be used to review trusted PRs."
  • - Anthropic, claude-code-security-review README

Three vendors' reviewers leaked their own API keys.

The exploit was a pull request title.

  • "Comment and Control", Johns Hopkins, April 2026
  • Anthropic's security reviewer, Google's Gemini CLI action, GitHub's Copilot agent
  • Anthropic rated it 9.4 out of 10. Bounty paid: $100.

The reviewer can be talked out of its findings by the thing it's reviewing

  • SEVRA-BENCH, Carnegie Mellon, June 2026: 1,062 real vulnerabilities, reversed and resubmitted with 15 persuasive framings
  • Claims the reviewer can't verify ("already approved upstream") push weaker reviewers' rejection rate below 35%
  • Verifiable lies get caught. Unverifiable stories don't.

The last reviewer is production

"I don't even call them pull requests. I call them prompt requests."

- Peter Steinberger, 27 Aug 2026

Shift the trust from review to validation

  • Charity Majors: "we need to increase trust at the other part of the process: tests, evals, and conformance testing"
  • Anthropic: an automated review of every past change would have caught one third of the bugs behind its incidents
  • Sarah Guo: "dark code" is "behavior in production that nobody can explain end-to-end"

Evals are your last line of defense

We can go faster,

at a higher level of abstraction

Someone has to decide it's their job

What to do on Monday

  • Stop reviewing pull requests line by line. Write down your blast-radius list instead.
  • Run two reviewers, not one. Tell them to assume the code is wrong. Feed your accept and reject clicks back in.
  • Ask for the transcript, not just the diff.
  • Treat the PR description as untrusted input.
  • Put your judgment into specs, tests, and evals. That's the harness. Then turn the agents loose on it.

Thanks

Follow me on BlueSky: @seldo.com

arize.com