AI Test Generation Tools: 8 of 76 Files Need No Mock

All ai test generation tools make the same offer: point it at a function, get a test back. The offer holds exactly as far as the function does. npm run check:testable classifies all 77 source modules in this repository by what a generated test would have to supply before its first assertion runs — a renderer, a request scope, a file tree, environment variables, or an HTTP transport. Eight modules need none of them. The other 68 need at least one, and the fixture the model invents to satisfy it is the part that decides whether the test means anything. If you are choosing the assistant that would write them, the assistants pillar is the decision above this one.

Key takeaways

• 8 of 76 exporting modules (11%) can be tested with no fixture at all, holding 22 of 173 callable exports and 20.6 kB of 247.2 kB of source.
• 45% of modules reach process.env — more than reach the network, the filesystem or a request scope combined, and 29 of the 34 inherit it through an import rather than reading it themselves.
• Six modules contain none of the five side conditions and inherit all of them. The newsletter API route is one: it writes to D1 through a helper, so a scan of its own text calls it pure.
• One module needs all five at once — the article page, which is also the highest-traffic route on the site.
• This repository has 0 unit tests and 36 committed checks, and every defect they have caught was caught by asserting on a build artifact, not by calling an exported function.

The short answer

Use ai test generation tools on the code that already takes values and returns values, and treat everything else as a fixture-writing task that happens to end in an assertion. The generation step is not where the cost is. On this codebase the model would write 8 tests that mean something immediately and 68 that mean whatever its invented fixtures mean, and reviewing 68 invented fixtures is not obviously cheaper than writing the tests yourself.

Then measure your own ratio before buying, because it is a property of your architecture and it varies enormously.

npm run check:testable
# → 77 modules, 173 callable exports, 0 test files
# → 8 of 76 modules (11%) need no fixture
# → 55 need a renderer, 34 need env, 16 need a transport

What a generator is actually pointed at

The script walks app/, components/, lib/ and db/, reads every export, and then follows the local import graph so a module inherits the side conditions of everything it imports. Types are counted separately, because a type export is not something a test can call.

The zero at the bottom is not an oversight this article is about to correct. It is the condition every one of these tools is sold into: a codebase with no suite, no runner, and no fixtures, where the pitch is that the tool supplies all three. What the numbers above decide is how much of that pitch is generation and how much is invention.

What ai test generation tools have to invent first

Each module is scanned for five conditions, then those conditions are propagated along the import graph. A module carrying none of them is importable by a bare node --test file — Node's built-in test runner, no dependencies — with no setup at all.

Two of those rows are more interesting than the headline.

env at 45% is the one nobody plans for. It beats the network, the filesystem and the request scope combined, and 29 of the 34 modules inherit it rather than reading process.env themselves. The chain is usually two hops and entirely ordinary: lib/schema.ts builds JSON-LD, imports lib/images.ts to make an absolute image URL, and that module reads NEXT_PUBLIC_R2_HOSTNAME at module scope. Importing the schema builder in a test with that variable unset does not throw — it produces URLs against a fallback hostname, and every assertion about them is quietly about the wrong thing.

The distribution is not flat. Eight modules have zero side conditions, 31 have one, 22 have two, and 15 have three or more. One has all five: app/[category]/[slug]/page.tsx, the article route, which renders React, reads the request scope, reads MDX off disk, reads env for the image host, and reaches a network call through the newsletter CTA. It is also the single most important route on the site.

Why the newsletter route looked testable

The first version of this script scored only what each file contains, and reported app/api/newsletter/route.ts as needing nothing. That route writes subscribers to Cloudflare D1. It is not testable without a transport by any reading.

The reason it scored clean is the reason this measurement is worth making at all: the route contains no fetch. The HTTP call lives one import away in lib/newsletter.ts, which is deliberate — the D1 access rules in this project put every query behind a single module so a future move to Workers is a one-file change. Good layering makes a file look pure and does nothing to make it testable.

app/admin/(panel)/newsletter/export/route.ts   inherits server, env, network
app/api/newsletter/route.ts                    inherits env, network
app/llms.txt/route.ts                          inherits dom, fs
app/sitemap.ts                                 inherits dom, fs
lib/admin/auth.ts                              inherits env
lib/schema.ts                                  inherits env

Direct scanning measures where the I/O is written. A test faces where it is reached from. Those are different sets, and the gap between them is six modules here — 8% of the surface, all of it in the layer a generator is most likely to be pointed at first, because route handlers look like functions with clean signatures.

The same correction dropped the headline from 15% to 11%. A tool that classifies testability by reading one file at a time will overstate it by the same mechanism, in the same direction.

The eight modules that need nothing

These are the ones where generate tests with ai is a straightforwardly good idea:

Two of them are worth the effort on their own. lib/pagination.ts parses a user-supplied ?page= parameter and computes bounds — the exact shape where a generated table of edge cases (zero, negative, non-numeric, past the last page) earns its keep in about ninety seconds. lib/admin/rate-limit.ts decides whether a login attempt is allowed, which is security-relevant, deterministic, and currently verified by nobody.

The rest are configuration objects and pure string functions. Tests for lib/site.ts would assert that a constant equals itself, which is the kind of coverage a generator produces abundantly and which nothing has ever caught.

What ai unit testing costs on a Next.js codebase

72% of modules need a renderer. That is not a Next.js problem, it is what an App Router site is: 55 of 76 exporting modules are .tsx files whose exports only exist once rendered. Testing them means a DOM implementation, a render helper, and a decision about what a component test is even asserting — Testing Library's guiding principles argue it should be what the user sees, which is a much larger commitment than "the function returned 4".

The mock is the specification. Vitest's mocking guide is explicit that mocking replaces the boundary with something you control. When a model writes both sides, the test verifies the model's belief about D1's response shape. That belief is not checked against Cloudflare by anything, and it will be confidently wrong in exactly the cases you wanted a test for. This is the same failure the review-scope measurement found in a different form — 9 of 76 modules hold 25.6 kB that reaches 613.5 kB through the import graph, and a reviewer shown only the diff never sees the part that breaks.

The honest cost model for generating a suite here is 8 tests worth reviewing on their merits, and 68 fixture-plus-assertion pairs where the review is of the fixture. Whether that trade is good depends on whether you would have written those fixtures anyway. For a service with a stable, documented API boundary, you would, and the generator saves real time. For a rendering site whose boundary is "Next.js", you mostly would not.

What we run instead

This repository has 0 unit tests and 36 committed check scripts, and the gap is not neglect — it is what the surface above pushes you into.

npm run check:anchors    # every toc href resolves to a real generated heading id
npm run check:mdx-attrs  # no braced attribute survives into content/
npm run check:failure    # ten seeded faults, scored on what each one printed

None of them imports an exported function. Every one asserts on an artifact the build produced: a compiled stylesheet, a prerendered page, a generated heading ID. That is a deliberate answer to the same question — how do you verify code whose behaviour only exists after a build — and it has caught defects a unit test could not have been written for. check:anchors catches a toc href that misses because rehype-slug strips the dot from CLAUDE.md; there is no function to call that would have told you that.

The counter-argument is real and this article should state it: those 36 checks all run when somebody types their name, and nothing in this repository runs any of them automatically. A test suite in CI has a property none of them has.

When to generate tests anyway

• The module is pure and the logic has edge cases. Pagination bounds, slug normalisation, a rate-limit window. Generation is fastest exactly where you would have been most bored.
• The boundary is documented and stable. If the API you would mock has a published schema, the invented fixture can be checked against it, and the test is worth what it claims.
• You are about to refactor. A generated characterisation suite over current behaviour is useful even if the assertions are dull, because the point is the diff after the change, not the assertions.
• Not for a component whose test would assert on markup. The generated selector will be brittle in a way that is invisible until a class name changes, and a broken selector fails green far too often.
• Not as a coverage number for a stakeholder. Nine of these modules would reach 100% line coverage on tests that only verify their own mocks, which is a worse state than 0% because it is reported as safety. The frontend surface audit reaches the same conclusion from the opposite direction: coverage is not verification.

Common mistakes we made

• We classified files instead of following imports. The first run called the newsletter API route pure. Fixed by propagating side conditions along the local import graph, which moved the headline from 15% to 11% — an error of a third, in the direction that would have flattered the result.
• We matched JSX with a regex that also matches generics. Array<string> in a .ts file satisfied the dom probe, so lib/newsletter.ts, lib/pagination.ts and lib/admin/session.ts were all reported as needing a renderer. Gating the probe on the .tsx extension removed 8 false positives and, embarrassingly, raised the pure count — lib/pagination.ts turned out to be one of the eight.
• We left a real module out of the graph entirely. lib/r2-hostname.mjs is imported by lib/images.ts and by next.config.mjs, and the scan only collected .ts and .tsx. The import resolved to a file that was not in the module map, so its side conditions silently contributed nothing.
• We assumed the network would dominate. It was the obvious candidate for the hardest fixture and it is fifth of five by module count. process.env beats it three to one, and nobody plans a testing strategy around environment variables.

What we are not claiming

No generator was run. Nothing here scores the quality of any tool's output. This measures the input all of them receive, which is the variable a vendor's demo repository is selected to make look good.

One architecture. Next.js 15 App Router, MDX, Cloudflare D1 over HTTP, Tailwind v4. A library with a narrow I/O boundary would score far above 11%, and that is the point — the number is a property of the codebase, so measure yours rather than borrowing this one.

"Needs a fixture" is not "untestable". All 68 are testable. They are testable at a cost, and the cost is a fixture somebody has to be right about.

The classifier is a regex, not a parser. A re-export counts as one export and a symbol re-bound through an intermediate is missed. Both undercount the surface, which pushes the pure share up, so the eight pure modules are listed by name above and can be checked by hand.

Conclusion

Run npm run check:testable's equivalent on your own repository before you buy an ai test generation tool, because the only number that matters is your fixture-free share and it is not the one in the demo. Point the tool at that share first — it is free value and it takes an afternoon. For the rest, decide whether you want a fixture you have to review, or a check that asserts on the artifact and needs no fixture at all. This project chose the second 36 times and has no unit tests, which is a defensible answer and not the only one. It also means nothing in the suite asserts on security behaviour, which is why the audit of what the agent actually wrote had to be its own instrument.

Frequently asked questions

Muhammad Kashif

Founder and editor of Devventa, covering AI coding assistants, Next.js and the modern AI development stack.