AI Code Review Tools: What the Diff Cannot Contain

Every ai code review tool in this category accepts the same input — a unified diff, plus a few lines of context — and that input carries a hidden claim: that a change can be judged from the lines it touched. On this repository it is true for exactly a third of the files. npm run check:review-scope builds the real import graph of app/, components/ and lib/ — 76 modules, 169 edges — and finds that 24 of them are imported by nothing, while 9 of them hold 25.6 kB of code whose changes reach 613.5 kB more. That is a 23.9x reading surface no diff of those files contains. If you are still choosing the tool that writes the code, the assistants pillar comes first.

Key takeaways

• 32% of modules here can be reviewed completely from the diff, because nothing imports them. For those files, more context is wasted budget.
• 12% of modules cannot be reviewed from a diff at all. lib/types.ts is 3.3 kB and reaches 38 of 76 files — half the codebase — through imports.
• Commit size does not predict review difficulty. A 36-line commit in this history reached 32 files; a 2,000-line one reached zero.
• 13 of 20 real commit subjects here are a single word, so the change's stated intent is not an input the reviewer has either.
• Some defects live in the relationship between files and are invisible to every diff of either one. This site shipped one: two fixtures naming the same tool differently, 370x apart.

The short answer

Choose an automated code review ai by what it is allowed to read, not by which model it runs. The useful axis is input width: can it fetch whole files, can it follow an import, can it run the repository's own checks, and can you set the depth per path. Model quality is real but it is downstream of whether the relevant code was in the prompt at all.

Then spend the budget unevenly. A third of your files need no context beyond the diff and a tenth need twenty times their own size, and treating both the same is how a review runs expensive and shallow at once. What actually fills a context window is the constraint that forces the choice.

npm run check:review-scope
# → 76 files, 249.2 kB, 169 import edges, 0 unresolved
# → 9 of 76 modules (12%) carry 25.6 kB whose changes reach 613.5 kB more — 23.9x

What every ai code review tool reads

Google's own code review guidance states the constraint plainly: "Usually the code review tool will only show you a few lines of code around the parts that are being changed. Sometimes you have to look at the whole file to be sure that the change actually makes sense." GitHub's Copilot code review documentation describes the same shape of input for the automated case.

So the question is not whether diff-scoped review is limited — everyone agrees it is — but by how much, on a real codebase. That number is a property of the import graph, and the graph is measurable with no network and no keys.

The script parses every .ts and .tsx file under app/, components/ and lib/, resolves each relative and @/ specifier the way the bundler does per TypeScript's module resolution rules, and builds the reverse edges. It exits non-zero on any specifier that does not resolve, because a missing edge makes the blast radius smaller than the truth — the one direction of error that would flatter the conclusion.

Twelve percent of the code carries the risk

Grouping every module by how many other files a change in it can reach:

The first row is the good news and it is a third of the codebase. Those 24 modules are route files, page components and leaves — nothing imports them, so a diff of one is the complete set of code that can change behaviour. An ai code review tool given only that diff is not handicapped at all, and paying for extra context there buys nothing.

The last row is the whole problem in 25.6 kB. Nine modules, a tenth of the file count and a tenth of the bytes, sit under 613.5 kB of dependent code — 23.9 times their own size. The specific offenders:

lib/types.ts is 3.3 kB of interface declarations and a change to it can reach half the site. Container.tsx is 1.1 kB — a max-width wrapper — with 14 direct importers. Neither is complicated. Both are places where "the diff looks fine" is a statement about 3 kB of a 90 kB question.

Twenty commits, and one that reached 32 files

The same script reads the real history. Twenty commits, median 18 files and 1,864 lines each, and the interesting result is how badly size predicts anything:

673855a changed 36 lines across three files and put 93.5 kB of dependent code in scope — a canonical-origin fix that touched lib/site.ts, which 15 files import directly. bba6b66 changed 20.5 kB of code and reached nothing, because all four files were leaves. A reviewer sizing effort by diff length gets both of those backwards.

Only 4 of these 20 commits touch the TypeScript surface at all. The other 16 are articles and measurement scripts, where the diff genuinely is the whole story — which is a real property of an editorial repository and a real limit on this sample, discussed below.

One more thing the reviewer does not get: 13 of 20 commit subjects on this history are a single word with no sentence in them. The stated intent of the change, which Google's guidance treats as the reviewer's starting point, is simply absent. That is a defect in this project's own habits and it costs an automated reviewer more than it costs a human, because a human can ask.

What a diff cannot contain

Some defects are not in any file. They are in the agreement between two files, and no diff of either one shows the disagreement.

This site shipped one on 2026-08-20. Two committed fixtures recorded a weekly download count for Cline: one had queried the package @cline/cli and recorded 198, the other had queried cline and recorded 78,781. Both files were internally correct. Both diffs were clean. The article that published the 198 passed every check the repository had. The defect existed only in the relationship between two fixtures, and it surfaced the next day when a comparison table put both numbers in the same row for the first time — a 370x disagreement between two of this site's own instruments.

That is the honest ceiling on this whole category. It is the same conclusion the frontend surface audit reached from the toolchain side: a class of defect with no verifier does not become visible by reading harder.

Choosing an automated code review ai

Rank on input width, in this order:

• Can it read whole files, not just hunks? This is the difference between covering 32% of your modules properly and covering all of them badly. If the answer is no, everything below is moot.
• Can it follow an import? One hop from lib/types.ts is 9 files; the transitive closure is 38. A tool that resolves specifiers can reach the code that actually breaks; one that does not is guessing from names.
• Can it run the repository's own checks? The widest-reach files here are type declarations, and tsc answers that question exactly. A reviewer that can execute npm run typecheck beats one that reasons about types in prose.
• Can you set depth per path? Uniform context is the wrong shape for a graph where a third of files need zero and a tenth need twenty times their size. Per-path budgets are the feature that makes the spend match the risk.
• Does it tell you what it did not read? Silence about scope is the failure mode. A tool that reports "reviewed 3 files, 32 dependents not read" is giving you the number this article had to measure by hand.

For teams the rollout question sits on top of all of that — who reviews the reviewer, and what happens on a disagreement. Security, compliance and rollout covers that half.

How to spend a review budget

• Compute your own hub list once. Ten minutes of import-graph parsing tells you which files deserve whole-repository context and which deserve none. Ours is five files long and did not match anyone's intuition, including the person who wrote them.
• Set depth by path, not by diff size. lib/** gets the transitive closure; route files get the diff. The measurement says a third of the codebase never needs more.
• Put cross-file invariants in scripts, not in review. Anything of the form "these two files must agree" is a check. The 370x disagreement above is now a one-line assertion that fails the build.
• Let the built-in reviewer do the first pass. Claude Code ships one, and five runs of it against six planted defects found all six between them for about a tenth of a dollar each — which makes it the cheapest place to spend the first slice of a review budget.
• Write a sentence in the commit subject. It is the cheapest input improvement available and 13 of our own 20 commits skipped it.
• Review the agent's long runs hardest. A single prompt on this project has produced 236 tool calls and 75 file edits with no human input in between, and that is exactly the diff nobody watched being written.

Common mistakes we made

• Measuring only the commits. Tempting because commits are what gets reviewed. The symptom was a median blast ratio computed from a sample of four, because 16 of 20 commits touch no importable code. The fix was to measure reach for all 76 modules instead, which is the number this article leads with.
• Assuming a big diff is a hard review. The 2,118-line commit in the table put 52.7 kB in scope; the 36-line one put 93.5 kB. Line count is a proxy for typing effort, not for reading effort.
• Trusting one-word commit subjects. Tempting when you are the only committer and you remember. The symptom shows up months later in exactly the artifact a reviewer needs most, and this repository has thirteen of them in its last twenty commits.
• Treating "no unresolved imports" as "complete graph". Static analysis cannot see a path built at run time. lib/mdx.ts finds articles by scanning a directory, so every file in content/ is a dependency this graph does not contain, and the script says so rather than implying coverage it does not have.

What we are not claiming

No tool is ranked here, and no defect count is produced. Measuring whether a given reviewer misses a given bug needs a labelled defect set, which this repository does not have. What is measured is the reading surface — the input every one of these tools accepts, and the thing a context limit actually constrains.

The commit sample is small and lopsided. Four of twenty commits touch TypeScript. That is honest for an editorial repository publishing sixty-odd articles, and it is why the headline number comes from the 76-module graph rather than from the commit table.

Reach is not severity. Nine modules reaching 613.5 kB does not mean nine modules are dangerous. Type declarations reach widely and break loudly; a CSS token reaches widely and breaks silently, which is a difference no graph captures.

Static imports only. Dynamic specifiers, environment-driven paths and directory scans create edges this measurement cannot see, all of which make the real blast radius larger than the printed one, never smaller.

Conclusion

Pick the ai code review tool that can be told what to read, then tell it — the transitive closure for your five or ten hub modules, the diff alone for the third of your files that nothing imports. Put every "these two files must agree" rule into a script, because no reviewer of either diff will ever catch it — and check that the script can actually fail, since one of this project's own checks reported a problem and exited zero for weeks. Then run npm run check:review-scope on your own repository: the hub list takes a minute to produce and it is the only input that makes the review budget match the risk. Point the resulting budget at the defects that have no checklist entry, because the named security risks are the ones agent-written code already handles.

Frequently asked questions

Muhammad Kashif

Founder and editor of Devventa, covering AI coding assistants, Next.js and the modern AI development stack.