The PR that broke me was forty lines against our auth service. The agent reviewer left three comments. Two flagged a missing trailing comma in a config object and the spelling of "occurred" in a log line. The third, verbatim: "LGTM, clean and well-structured." The diff it called clean and well-structured concatenated a user-supplied org_id straight into a raw SQL WHERE clause. Plain string interpolation. In 2026.
So my first honest verdict on letting an agent review pull requests: mistake. A confidently wrong intern with infinite stamina, flagging commas while waving a SQL injection through the gate. I kept it anyway, because I changed one structural thing and it stopped being a liability. This is the case against, then the case for.
The miss isn't random, which is what makes it dangerous
I assumed the agent whiffed on the injection because it was having an off day. It wasn't. The failure is structural, and structural is worse than random — predictable failures are the ones you can build defense against, and we'd built none.
Veracode's Spring 2026 GenAI Code Security update ran 100-plus models — the GPT-5 line, Gemini, the Claude 4 family — and found that roughly 45% of AI-generated code ships with a known security flaw. That number sat flat across two years of vendor slides each promising the corner had been turned. The damning part is the spread by vulnerability class. SQL injection gets handled correctly something like 80% of the time. Cross-site scripting: closer to one in seven. Log injection, worse.
Sit with that shape. The model is competent at the famous vulnerability and nearly blind to the unglamorous ones, because the famous one dominates the training data and the tutorials and the Stack Overflow answers, and the unglamorous ones don't. An agent reviewer inherits that exact profile. It is chatty and confident about the bug it has seen ten thousand times and silent about the one it hasn't — and silence reads as approval. Syntactic correctness clears the bar nearly every time while secure-code rate sits barely above a coin flip. The output looks finished, and "looks finished" is exactly what disarms a reviewer's skepticism. Clean is not safe. The agent had learned to produce clean and nothing underneath it.
Volume turned a nuisance into a structural risk
You could shrug this off when agent PRs were a novelty. They're the median now. GitHub's Copilot review has cleared tens of millions of reviews with roughly 10x growth inside a year; more than one in five code reviews on the platform now run through an agent. When the machine writes the diff, the machine reviews the diff, and a human glances at a green check, nobody read the code. Three actors in the loop and zero comprehension events.
The rubber-stamp problem is the one nobody measures until it bites. "Clean migration, CI green, LGTM" on a database migration large enough to take the site down — approved in the time it takes to read the title, with no evidence anyone opened the diff. I measured ours after the auth incident and it was ugly. The tell is brutal and arithmetic: if your average review time on agent-authored PRs matches your average on human PRs, you aren't reviewing, you're approving with extra latency.
Here's the kicker I didn't see coming. A 2026 study — More Code, Less Reuse — found reviewers feel more confident approving agent-authored code, even as that code carries measurably more redundancy and tech debt per change. The clean surface buys trust it hasn't earned, and the trust compounds: each rubber stamp makes the next one easier. Same psychology that walked my injection through the gate.
The lack of determinism is the hidden killer
What finally convinced me the review process was broken, not just the model, was scope drift. I ran the same agent twice on the same refactor with the same prompt. One pass touched 2 files. The other touched 11. Same input, 5x the blast radius.
Unstructured review has no answer to this, because it has no fixed frame. It reviews whatever shape the diff happens to arrive in. If the diff sprawls, the review sprawls with it and loses the plot; if the diff is tight, the review invents nitpicks because there's nothing else to chew on. "Review this PR" isn't an instruction, it's a mood. Moods don't have a pass condition. That was the actual bug — not the model, the prompt. We'd automated the judgment and left the criteria implicit, which is the one place automation has nothing to grip.
The one change: a diff-scoped checklist with a hard gate
I stopped asking the agent to "review this." I gave it the same ordered list of failure modes, run against the diff and nothing but the diff, every single time, ending in a binary verdict that blocks merge. Pass or fail. Not a comment thread — a gate.
The diff-scope is half the fix. Reviewing only what changed, in the same order regardless of how the diff is shaped, kills the nitpicking (the comma was never in scope) and kills the scope-drift miss (the checklist doesn't care that the diff arrived weird; it walks the same path anyway). The hard gate is the other half. It converts "is everyone comfortable with this?" into "did the diff pass?" — a question with an answer instead of a feeling.
Roughly what's in ours now:
GATE: fail-closed. Any FAIL blocks merge.
1. CI gaming — tests removed, lint skipped, coverage weakened? → hard FAIL
2. Reuse blindness — reinvented a util that exists in repo? → FAIL
3. Hallucinated correctness — trace critical paths end to end:
off-by-one, missing permission check, race that compiles fine
4. Untrusted input — every external value: sanitized? token over-scoped?
raw string into SQL/shell/HTML? → hard FAIL
5. Scope discipline — >5 unrelated files, no one-line purpose,
no plan? → DECLINE, don't review
Item 4 would have caught my auth PR — not because the model got smarter overnight, but because "every external value reaching a sink" is now a question it has to answer out loud and fail on, instead of a thing it might mention if the mood took it. The gain wasn't capability, it was obligation. Item 5 earns its place too: letting the reviewer refuse an oversized, purposeless diff is what stops the scope-drift garbage at the door, instead of pretending to review eleven files it can't hold in its head at once.
Optimize for recall, then make recall block
Pick recall over precision, on purpose. False positives are noise you triage; false negatives are the thing that pages you at 2am. I'll take ten false alarms over one shipped injection every day of the week. But — and the original mistake lived in exactly this gap — a high-recall reviewer with no gate is just ignorable noise with good intentions. You need the recall and the hard fail, or you've built a very thorough comment generator that the whole team learns to scroll past inside a week.
The tool you pick under that gate matters less than the threads argue. In one planted-vulnerability test, Copilot's reviewer caught all five seeded flaws including the SQLi and an auth bypass; another well-known scanner missed three. One vendor runs the lowest false-positive rate; another catches the most and is noisy for it. Pick your spot on that curve — it barely matters. Every one of them rubber-stamps without a scoped checklist and a gate wrapped around it. The structure is the product. The model is a component you can swap.
Did it earn the seat
Anthropic reports their structured, multi-agent Claude Code review pushed substantive review comments from 16% to 54% of PRs at around 20 minutes average, with under 1% of findings flagged incorrect — their numbers, not mine, but the shape matches what I watched happen on our repo. Our agent now runs as a prerequisite gate: it does the mechanical pass, fail-closed, before a human spends judgment on the parts that actually need a human. It is not the approver. It is the thing that has to go green before approval is even on the table.
That's the whole turn, and it's smaller than it sounds. The agent didn't earn its seat by getting smarter. It earned it the day I stopped trusting its judgment and started trusting a checklist it isn't allowed to skip. Give it vibes and it'll flag your commas and miss your injection. Give it a scoped list and a gate that says no, and it becomes the most tireless junior reviewer on the team — which, it turns out, was the only job I needed it to do.