A diff from an agent can pass every test, satisfy the linter, and still be wrong in a way none of that catches — because tests check the behavior someone thought to write a test for, and an agent under-specified or over-scoped the change in a way that only shows up on inspection. The mistake is treating "CI is green" as the review. It's necessary and it's not sufficient, for the same reason it was never sufficient for a human's diff either — it just fails differently for an agent, in patterns specific enough to check for directly instead of re-reading the whole diff line by line hoping something jumps out.
Agents tend to "improve" adjacent code while they're in the file for an unrelated reason — rename a variable, reformat a block, refactor a helper it didn't need to change. Each of those is individually plausible and collectively makes the diff impossible to review by intent, because now there are two changes tangled into one and no way to tell which lines were the fix and which were incidental. Check the diff's line count against the task's description before reading a single line of logic: a one-line bug fix that touched forty lines across six files is a diff to send back unreviewed, not one to start reading.
The riskiest lines in any diff are the ones with a - in front of them, and they're the easiest to skim past because attention naturally goes to what's new. An agent asked to "simplify" or "clean up" a function will sometimes remove a null check, an error branch, or a retry that looked redundant in isolation but existed because of an incident six months ago that isn't recorded anywhere the agent could see. If a diff removes error handling, a conditional, or a guard clause, that deletion needs its own justification — "it looked unreachable" is not one, since unreachable-looking code guarding a real edge case is exactly what postmortems are made of.
Given an open-ended instruction, agents default toward generality — a config-driven strategy pattern for a branch that will only ever have two cases, a plugin interface for a call site with one caller. It's not wrong, exactly; it's solving a problem that doesn't exist yet at the cost of a problem that does: more code to hold in your head for the same behavior. If the abstraction has exactly one implementation and no second use case in sight, that's the tell — ask for the direct version instead of the general one, the same way you would with a human PR that reached for a design pattern too early.
| Signal | What it usually means |
|---|---|
| New code path with no corresponding test change | The happy path may work; the edge cases were never exercised, agent or human |
| Error messages or log lines that don't match the actual failure condition | Copy-pasted from a similar branch and not re-checked against this one |
| A dependency added but never imported, or imported but unused | Reverted or abandoned an approach mid-change without cleaning up after it |
| Comment describing behavior the code next to it doesn't have | Code changed after the comment was written; neither was reconciled |
| Confident claim in the summary that doesn't match the diff | The summary describes what was intended, not necessarily what happened — verify by reading the diff, not the description of it |
Not every diff needs the same scrutiny — a typo fix in a comment and a change to payment retry logic are not the same review. Reserve the full checklist above for changes that touch money, auth, data deletion, or anything running unattended without a human in the loop; a low-risk cosmetic change doesn't need it. This is the same trust-boundary judgment as deciding what a headless pipeline is allowed to do — the review effort should scale with the blast radius of being wrong, not be uniform across every diff regardless of what it touches. A diff that adds a new tool for an agent to call is worth the same question up front — see designing tools for agents to call for what makes a repeat call safe by construction. And a diff review only ever covers the code that shipped, not the runtime decisions made after — see building a reconstructable audit trail for the log-time half of that same accountability question. A diff review also only catches what one reviewer notices in one pass — for the same question asked systematically across every change over time, see regression testing for agent behavior.
None of this is specific to any one agent or tool — it applies just as much to a diff a junior engineer wrote quickly. What's different with an agent is the failure modes cluster more tightly around these five patterns, so a targeted checklist catches more per minute spent than a general "read it carefully" pass does.