Blog

You're reviewing the wrong thing.Copy link

Two posts, a thousand-plus reactions each, spent the week fighting over whether engineers should still read AI-generated code. Both camps are reviewing the code. The review that decides whether the software is right needs to point at the chain that produced it.

There's a proper scrap running on LinkedIn this week about whether you should still read the code your AI writes.

Santiago Valdarrama, an ML educator with a large following, started it: he is "officially done reading AI-generated code". Two weeks since he looked at any of it. When he was still reviewing, his complaints had dwindled to style; the more code he generated, the harder it got to track every line; and his time, he says, is better spent "designing ways to verify that the overall system works" than looking at the code. Eleven hundred reactions and a hundred and forty comments when I read it.

Vlad Mihalcea, a Java Champion, came straight back the other way: if an LLM writes code you can't understand, that's your cue to upskill, because reviewing AI-generated code is about to become "a mandatory requirement for software engineers". Another thousand reactions. The comment sections under both are the industry arguing with itself.

Faysal Hassani, a director of technology development, added the sceptic's footnote: the people announcing they no longer read the code tend to be selling coding tools, building small apps with no real complexity, or not the ones holding the maintenance bill when it comes due.

I've spent the last year running coding agents on real production work, and I don't recognise my own position in either camp. Both of them are arguing about how much attention the code deserves. I think the code is the wrong thing to be reviewing.

Both camps are right, which settles nothingCopy link

Give both their due, because each post carries something true.

Santiago's arithmetic is honest. When the volume of generated code passes what you can hold in your head, and the only defects your reading still catches are stylistic, the reading has stopped being a control and become a habit. He's also honest about what replaces it, and I'll come back to that sentence, because it's the best one in the whole debate.

Vlad's warning is honest too. Somebody has to understand the system when it breaks at 2am, and "the AI wrote it" is not a sentence that survives an incident review. Skipping understanding entirely is how you end up operating software nobody in the building can reason about. Faysal's three categories are the same warning with names attached: not reading the code is cheap when someone else owns the consequences.

But look at what the two positions share. Read every line, read no lines, read the risky lines: all of it answers the same question, which is how hard to stare at the diff. Nobody in either comment thread asks what the staring is supposed to establish.

Code review was always two jobsCopy link

Code review has been doing two jobs at once for as long as I've been in engineering, and we never bothered separating them because one person did both in one pass.

The first job is construction: the race condition, the resource leak, the query inside the loop, the error path that swallows the failure. That evidence lives in the diff, and reading the diff is how you find it.

The second job is intent. Is this what we asked for? Does "done" here mean what the business meant by done? That evidence never lived in the diff. It lived in the reviewer's head, carried in from the ticket, the stand-up, the conversation by the kettle. Code review checked intent only because a person who happened to know the intent happened to be looking.

AI generation speed pulls the two jobs apart. The construction check scales tolerably: linters, type systems, static analysis, a second model reading the first model's output. The intent check doesn't scale with it, because the reference it needs was never written down anywhere a machine, or a colleague, could reach. FICO is hiring a Director of AI Harness Engineering as I write this, and the hiring post names the problem exactly: agents generate code faster than humans can review it, so the bottleneck shifts to verification and trust. Their equation for the discipline is Agent = Model + Harness. When a company that size creates a director role for the verification bottleneck, the question has already left the comment section.

So Santiago is right that line-reading is finished as the primary control, and Vlad is right that somebody has to be able to check the work. The way through is that the check has a better target than the code.

Review the chainCopy link

On an AI-assisted change, four questions decide whether the software is right, and none of them start with the diff.

  1. The requirement. Where is the ask written down? Not the prompt. If the ask lives in a chat thread that scrolled away last Tuesday, no amount of code reading will tell you whether the code is right, because there's nothing for right to mean.

  2. The acceptance criteria. What did "done" mean before the code existed? Criteria written after implementation describe what the code does, not what it was supposed to do. The definition of done has to precede the code it judges.

  3. The test bindings. Which tests bind to which requirement? A green suite on its own proves the code agrees with itself, especially when the same agent wrote both halves moments apart. The suite means something when you can walk from a failing test back to the specific requirement it protects, and from a requirement forward to the tests that cover it.

  4. The verification. Did any of it touch the surface users touch? Tests passing against mocks are a statement about the mocks. If nothing drove the actual app the way a user would, the green run describes the test environment, not the product.

Ask those four before anyone opens the diff and the line-reading debate mostly dissolves. With the chain in place, reading the code becomes what Vlad wants it to be: a skilled construction check against a reference you can point at, applied where the risk justifies it. Without the chain, reading the code is all you have, and Santiago's arithmetic says you'll drown doing it.

The best evidence in the whole thread came from the smallest post in it. Patrizio Pezzilli, who works in AI engineering at UniCredit, suggested an exercise you can run this week: sort your last thirty defects into specification failures and implementation failures. If most turn out to be things nobody specified properly, then generating code faster isn't your constraint, and buying more of it won't help. Sixteen reactions. It deserved the eleven hundred.

Review the chain, not the diff: four questions decide whether AI-assisted software is right, and none of them start with the code A feed-legible infographic in three parts. Top: eyebrow week 13 reviewing the wrong thing, and a right-hand eyebrow four questions on the chain. Big serif headline: Review the chain, not the diff, with the word chain in signal red. Subtitle: four questions decide correctness on AI-assisted work, the chain from ask to proof. Below, four signal-stroked cards in a row. Card one, chip REQUIREMENT, big numeral 01: WHERE IS THE ASK, not the prompt (the word ASK is bolded). Card two, chip ACCEPTANCE CRITERIA, big numeral 02: WHAT DID DONE MEAN, before the code (DONE is bolded). Card three, chip TEST LINKS, big numeral 03: WHICH TESTS BIND WHERE, requirement to case (TESTS is bolded). Card four, chip VERIFICATION, big numeral 04: DID YOU TOUCH USER SURFACE, not the mocks (USER is bolded). Below the cards, an evidence band. Eyebrow: the debate, read every line versus read no lines. Headline: Both camps are staring at the diff. Attribution: the reference they need was never written down. Closing line at the bottom, in signal red: the review has moved one layer up. week 13 · reviewing the wrong thing four questions on the chain Review the chain, not the diff. four questions decide correctness on AI-assisted work, the chain from ask to proof Question 01: Where is the ask written down? Not the prompt, the requirement. If the ask lives in a chat thread that scrolled away, no amount of code reading will tell you whether the code is right. Requirement 01 WHERE IS THE ASK not the prompt Question 02: What did "done" mean before the code existed? Acceptance criteria written after implementation describe what the code does, not what it was supposed to do. Acceptance criteria 02 WHAT DID DONE MEAN before the code Question 03: Which tests bind to which requirement? A green suite alone proves the code agrees with itself. It means something when you can walk from a failing test back to the requirement it protects, and from a requirement forward to the tests that cover it. Test links 03 WHICH TESTS BIND WHERE requirement to case Question 04: Did the verification touch the surface users touch? Tests passing against mocks are a statement about the mocks. If nothing drove the actual app the way a user would, the green run describes the test environment, not the product. Verification 04 DID YOU TOUCH USER SURFACE not the mocks the debate · read every line vs read no lines Both camps are staring at the diff. the reference they need was never written down the review has moved one layer up Review the chain, not the diff (narrow layout) Same content stacked vertically. Four signal-stroked cards top to bottom: WHERE IS THE ASK not the prompt (ASK bolded), WHAT DID DONE MEAN before the code (DONE bolded), WHICH TESTS BIND WHERE requirement to case (TESTS bolded), DID YOU TOUCH USER SURFACE not the mocks (USER bolded). Below, evidence band: both camps are staring at the diff, the reference they need was never written down. Closing line: the review has moved one layer up. week 13 · four questions on the chain Review the chain, not the diff. four questions on AI-assisted correctness Question 01: Where is the ask written down? Not the prompt, the requirement. Requirement 01 WHERE IS THE ASK not the prompt Question 02: What did done mean before the code existed? Acceptance criteria 02 WHAT DID DONE MEAN before the code Question 03: Which tests bind to which requirement? Test links 03 WHICH TESTS BIND WHERE requirement to case Question 04: Did the verification touch the surface users touch? Verification 04 DID YOU TOUCH USER SURFACE not the mocks the debate · every line vs no lines Both camps are staring at the diff. the reference was never written down the review has moved one layer up

The declared interestCopy link

This is the problem I've spent the last year building against, so I'll say so plainly and keep it short. RCF is my answer: requirements, user stories and acceptance criteria living in the repository with IDs on everything, test suites generated from the criteria before implementation, and tooling that walks the links and fails loudly when one is missing. A mechanical chain from the ask to the proof. It's the four questions above, answered by a validator rather than a meeting, and it has put real software into production for me this year. But you don't need my tooling to run the argument. The four questions work from a checklist on Monday morning.

Which brings me back to Santiago's best sentence: his time is better spent designing ways to verify that the overall system works. He's right, and I'd go further: that design is the job now. A verification design needs a reference to verify against, and the reference is the one artefact neither camp is reviewing, because it's the one the industry keeps not writing down. The code never held it. IMHO the review that matters has moved one layer up, to the chain that produced the code. The diff was never where the trust lived anyway.

Barry