Your Review Gate Was Built for a Defect That No Longer Shows Up

Output is up. Cycle time is down. Approval latency has dropped so far that the review queue stopped being a topic in the staff meeting. And somewhere in the same quarter, a competent engineer started needing an afternoon to answer a question that used to take ten minutes.

Nothing on the dashboard explains that. Defect rate is flat, coverage held, and nobody has shipped an outage worth a postmortem.

The instinct is to look harder for bad code. That instinct is pointed at the wrong thing, because the defects everyone is scanning for are precisely the ones the gate already catches.

AI generated code quality is not mainly a hallucination problem. Conventional pre-merge review was calibrated to catch visible error, and generated code that is locally correct but poorly fitted to its system can clear that gate and accumulate underneath it.

There is data on the visible half. CodeRabbit's December 2025 State of AI vs Human Code Generation Report found AI-co-authored pull requests carried roughly 1.7x more issues than human-only ones, at escalated severity, with readability issues up more than 3x. Every one of those is a detected issue. That is the point, and it is also the limit: a detector reports what it detects, and in that study even the authorship split was inferred from co-authorship signals rather than confirmed. In Augment Code's 2026 survey of 219 engineering leaders, 55% said they were concerned about losing shared understanding of the codebase, and 39% were worried about shipping with confidence. Those are self-reported concerns rather than measured outcomes, and they point at a question the first report cannot answer.

The pull request that passes everything

The shape I keep running into looks like this, as concrete as I can make it without pointing at anyone.

A service needs retry behavior on an outbound call. The change is 61 lines. It adds a small RetryPolicy class inside the module that needed it, with exponential backoff and a jitter factor. Tests come with it, four of them, covering the backoff math and the exhaustion path. Naming matches the house style guide. The linter is silent. Coverage ticks up by a tenth of a point. A reviewer approves it in eight minutes with one comment about a variable name.

Every judgment in that paragraph is correct. The code does what it says, and the tests test the thing. The reviewer was not lazy, and eight minutes is a defensible amount of time to spend on 61 legible lines that do one obvious job.

What the diff does not show, because a diff cannot show it: a shared platform package already carried a retry abstraction, written two years earlier, with the backoff curve the infrastructure team tuned against the actual failure profile of that dependency. The new class solves this endpoint. The old one solved the class of endpoints. Nothing in the pull request references the old one, because nothing in the pull request needed to.

That's the whole case. It is unremarkable, which is the reason it matters.

What the gate was actually built to see

Pre-merge review is not a general-purpose quality instrument. It is a set of specific detectors, each aimed at a defect signature someone once got burned by.

Lint catches style drift and a narrow band of correctness. Type checking catches shape mismatches. Tests catch behavioral regression, but only against the cases someone thought to write. Coverage catches untested paths, and static analysis catches known-dangerous constructs. Human review catches the residue, and human review is where architectural judgment has always lived, in the form of a senior engineer who happens to remember the platform package.

Run the retry PR through that stack and every gate returns the correct answer. Not a false negative anywhere. The gate reported accurately on the questions it was built to ask, and none of those questions was "does this belong here, given what already exists."

This is the part that gets misread as a tooling gap. You can point a detector at a defect signature only after someone has articulated the signature, and "duplicates an abstraction that lives three packages away" has never had a clean signature at the diff level.

The signal review was tuned to is getting weaker

This is the argument. I want to bound it before making it, because the sloppy version of this claim is everywhere, and it's wrong.

Under human authorship, visible error and architectural weakness have tended to travel together. The same conditions produce both: time pressure, unfamiliarity with the system, a junior engineer working past the edge of what they've been shown. An engineer who doesn't know the platform package exists is also, usually, an engineer whose code has other tells. Rough naming. A test that asserts the implementation instead of the behavior. Something a reviewer catches, and while catching it, notices the larger problem.

That correlation was a tendency, never a law. Human-written code could always be locally correct and architecturally harmful, and any senior engineer can produce three examples from memory. Skilled, rushed people ship clean-reading code that is wrong for the system all the time.

So the claim is not that generation created a new category. The hypothesis is narrower: generation weakens that correlation further, because it makes plausible, locally correct, poorly fitted code much cheaper to produce in volume. The model writes idiomatic code by construction. Unless the system's history is retrieved for it, by repository search, dependency inspection, or an ADR someone points it at, it works from the local context alone. It will produce something defensible for the file it's looking at, with clean naming and reasonable tests, far more often than not, without the accompanying tells that used to travel with unfamiliarity.

The conditions under which this holds are worth naming, because they're also the conditions under which it doesn't. It holds where review is pre-merge, diff-scoped, and has no explicit system-fit question. It weakens where architectural review is a distinct step, where dependency direction is enforced mechanically, where an ADR process exists and is actually consulted, where ownership review routes changes to whoever holds the abstraction, or where integration testing exercises the seams. Those mechanisms detect this class today. Later maintenance detects it too, at a much worse exchange rate.

I used to read this as a volume story: more code, more review, more misses, roughly proportional. It isn't proportional, and that's what changed my mind about it. Volume alone would raise the miss rate on defects the gate can see. What actually shifted is the mix of what arrives at the gate, and the fraction of arriving work that carries no signal the gate was designed to read.

That distinction is why this argument is not the review-cost argument, which is about what reviewing more code costs you. This one is about what clears review and then compounds. It also sits downstream of the bottleneck relocating: once generation stops being the constraint, what survives the next constraint becomes the thing worth measuring.

Architecturally inert, defined so a reviewer can use it

An abstraction nobody can apply is not worth introducing, so here is the operational version.

Code is architecturally inert when it works and contributes nothing to the structure it lands in. Four observable conditions, any of which a reviewer can check against the repository rather than against taste:

• It duplicates an existing abstraction instead of reusing it, where the existing one is reachable and not deprecated.
• It solves the instance rather than the class, in a place where the class is already named somewhere else.
• It adds a layer that no caller asked for, where the indirection has exactly one implementation and one consumer.
• It is individually reviewable and collectively incoherent: each unit passes on its own, and the set of them has no single shape.

The retry PR satisfies the first two outright and the fourth once its siblings arrive.

Same obligation applies to the rest of the vocabulary in this piece. Code maintainability and architectural coherence are only worth invoking if you can say what would be observed. Coherence here means: for a given responsibility, one place owns it, and that place is discoverable from the call site. Structural drift means: the count of responsibilities with more than one owner is going up over time. Both are countable, and neither is on a default dashboard.

Every entry in the right-hand column is a gate correctly reporting that the thing it measures is fine.

Compliance is not fitness

The strongest of the ranking pieces reviewed for this argues that you should encode your standards so agents comply with them: version the instructions, review them, treat them as infrastructure. That is correct, and it is worth doing regardless of anything in this article. Compliance is a distinct property, and encoding it takes work.

It's also a different property from fitness, and the gap between them is where the retry PR lives. AI coding standards can be fully satisfied by code that is wrong for this system. Naming conventions, error handling, documentation, module layout: all checkable, all satisfied, none of them load-bearing on the question of whether this abstraction should exist here.

Several architectural properties are encodable, and I want to concede this clearly because the overclaimed version of my argument is easy to knock down. Dependency direction is encodable. Layering violations are encodable. Coupling thresholds, forbidden-interface rules, module-boundary enforcement: ArchUnit does this, dependency-cruiser does this, import-boundary linting does this, and teams run them in production today. If you are not running any of them, that is a gap with a known fix, and it will catch a meaningful slice of what I've described.

The defensible claim is narrower than "rules can't see architecture." It's that no encodable rule exhaustively determines architectural fitness, because fitness is a judgment about this codebase's trajectory. Whether a second retry abstraction is a duplication problem or a legitimate divergence depends on where the system is going, which team owns the dependency, and whether the platform version is being deprecated. A rule can flag the duplicate. Only a person who holds the trajectory can say whether the duplicate is wrong.

Partial measurement is real, and conceding it is the honest position. It also raises the stakes on the residue, because the residue is exactly the part that requires the scarcest reviewer you have.

Why the dashboard stays green through all of this

Ask what would have to move for the retry PR to register as a problem in a normal engineering review.

Defect rate won't move, because the code isn't defective. Velocity won't move, or it will move the right way. Review throughput improves, because 61 legible lines approve faster than 61 tangled ones. Coverage improves, cycle time improves, and escaped-defect count is unchanged. Every instrument on the panel reports health, accurately, about the thing it measures.

Some metrics do bear on this. Duplication percentage exists in most static-analysis suites, and so do coupling and cohesion metrics. Dependency-graph tools will show you a second edge where there should be one. The problem is that these live one layer below the default dashboard, they are rarely on the review that leadership actually sees, and none of them is trended against a baseline anyone agreed to.

Green metrics here tell you the instrumentation was built to ask a different question.

I keep seeing the same failure, and it always looks like a win first. A team ships a quarter's worth of endpoints in about six weeks. Small PRs, tests throughout, same-day approvals, the kind of stretch people screenshot for the board deck. Then someone picks up a routine bug in the third of those endpoints and finds retry logic in four places, written four ways, each one correct on its own, none of them the platform version. That platform version makes five. Fixing the bug takes a day. Working out which of the five the fix belongs in, and whether changing it breaks the other four, takes the rest of the week.

Nothing in that story shows up as a defect. It shows up as a competent engineer being slow, which is the most expensive thing to misdiagnose, because the obvious reading is a people problem and the actual reading is a standards problem.

This is also where the operating-model component most people skip becomes load-bearing. Incentives and performance measures reward what they can see. Approval latency is visible, so it gets optimized. Architectural coherence is not on the panel, so no one is accountable for it, and no one is behaving badly. They're behaving exactly as the measurement design rewards them to behave.

What actually has to change, and who can change it

Not a tooling recipe. The three things that matter here are decisions a CTO or VP Engineering already controls. Not one of them requires new tooling spend. All three cost reviewer and owner time, which is the scarcer budget.

Change the question review asks. Most review protocols implicitly ask "is this code correct." Add an explicit second question for changes that introduce an abstraction: does this responsibility already have an owner in this system, and if so, why is this one different. That's a workflow and handoff change, not a tooling change, and when the responsibility has an obvious owner the question resolves in one line.

Change who reviews what. Diff-scoped review by whoever is available is fine for changes that don't touch structure. Changes that add an abstraction, a layer, or a dependency edge need routing to whoever owns the surrounding structure. That's a review and control standards change, and it's the one most orgs already have the org chart for and haven't wired into the workflow.

Change what gets counted. Put two counts on the same review as velocity. Duplication percentage already exists in most static-analysis suites. The second, the number of responsibilities carrying more than one implementation, doesn't come out of a tool: one owner names the responsibility list, and the count only means anything against that same list over time. Not as a gate. As a trend with a baseline, so that drift becomes visible before it becomes archaeology. That's the incentives and performance measures component, and it's the one that makes the other two stick.

The through-line is that all three are about what your delivery org treats as an inspectable artifact. Review patterns, meaning what gets reviewed, by whom, and at what depth, are one of the six artifact classes in the Shift Harness Artifact Test, and they're the class most directly exposed by generated code. Everything above is a specification of that one class.

The retry PR will get approved again next week. It should, under the current standard. The question worth taking into your next engineering review is not whether your gate is working. It's what your gate was asked to look for, who decided that, and how long ago.

Key takeaways

• Conventional pre-merge review reports accurately on the defect signatures it was built to detect; architecturally poor code that is locally correct produces none of those signatures.
• The hypothesis worth testing is that generation weakens the historical correlation between visible error and architectural weakness, not that it created a new defect category.
• "Architecturally inert" is checkable: duplicates a reachable abstraction, solves the instance where the class is already named, adds a layer with one caller, individually reviewable and collectively incoherent.
• Dependency direction, layering, coupling and duplication are encodable and enforced in production today; no encodable rule exhaustively determines fitness, because fitness is a judgment about trajectory.
• The measurement gap is a dashboard-design problem, not an absence of metrics. Duplication and multi-owner responsibility counts exist and are rarely trended.

AI Transparency Notice: This article and its accompanying images were created with the assistance of generative AI. The author directed the content, contributed the underlying ideas and analysis, and reviewed the final publication.

Frequently Asked Questions