Context Window in AI Coding: What Actually Fills It

The context window in AI coding is filled by two things you did not type: the session floor the tool loads before it reads your prompt, and the files it opens on your behalf. On this machine, on 2026-08-18, a headless Claude Code turn with completely empty input reported 31,979 tokens of context, and one committed lockfile from this repository added 139,640 more. This article measures what a file actually costs, shows why the usual four-characters-per-token estimate undercounts every file we tried, and gives the method so you can run it on your own code.

Key takeaways

• A token is 3.4 bytes of Markdown, not four. Measured across nine real files, the ratio ranged from 2.213 bytes per token for JSON to 3.948 for MDX prose — a 1.78x spread that no single constant covers.
• The four-chars-per-token estimate undercounts every file we measured. It reports this repository's per-session floor as 2,034 tokens where the measured ratio gives 2,393, a 17.6% shortfall in the direction that hurts.
• 31,979 tokens are gone before you type. That is the system prompt and tool definitions, 16% of a 200,000-token window, and it is invisible in the interface.
• package-lock.json alone is 139,640 tokens — 69.8% of a 200k window, from one file nobody chose to read.
• Run order changed a measurement by 19%. A cold first turn writes the cache rather than reading it, and comparing against it moved README.md from 3.399 to 4.062 bytes per token.

The short answer

Budget your context window by measuring the files, not by dividing bytes by four. Source code tokenises worse than prose, structured data much worse, and the error compounds across a session until compaction arrives earlier than you planned for.

If you want the mechanics of the tool that spends this budget — permissions, subagents, what gets loaded and when — the complete comparison of AI coding assistants is the pillar this sits under, and what slows Claude Code down is the same problem observed from the other end.

How we measured what a file costs

Claude Code reports its own token accounting under --output-format json. The usage object carries three input figures, and their sum is the context the request actually carried:

cat lib/mdx.ts | claude -p "Reply with exactly: ok" --output-format json --effort low --model haiku
# → input_tokens + cache_creation_input_tokens + cache_read_input_tokens = 34,620

The trick is the subtraction. Run the same four-word prompt with empty input, and everything that is not the file — system prompt, tool definitions, CLAUDE.md — is identical between the two runs and cancels:

tokens(file) = (input + cache_creation + cache_read)          ← with the file on stdin
             - (input + cache_creation + cache_read)          ← with empty stdin

That is the whole instrument, and it is committed as scripts/check-context-fill.mjs in this repository:

npm run check:fill

The evidence that the floor really did hold still is in the output: every run that carried a file reports the identical cache_read figure of 26,138. If that number moves between runs, the difference is not attributable and the script fails rather than printing a table.

A token is not four characters

Nine files from this repository, measured on 2026-08-18 against Claude Code 2.1.234:

Every file came in under four. The four-characters-per-token figure is a reasonable approximation for running English prose, and source code is not running English prose — it is dense in punctuation, brackets, quotes and short identifiers, all of which the tokeniser spends separately.

Two things follow. The first is that the error is not random: it is one-directional, and it always tells you the file is smaller than the model will charge for. The second is that no single constant fixes it. JSON and MDX are 1.78x apart, and a session that reads both gets a different average depending on which it read more of.

This one is not theoretical for us. scripts/check-context-weight.mjs on this site divided by four, and it reported this repository's per-session floor as 2,034 tokens. At the measured Markdown ratio the same 8,137 bytes are 2,393 tokens — 17.6% more. The constant has been changed to 3.4, the header now names the date and the file it was measured from, and check:fill fails if it ever drifts back outside the measured range.

The session floor loads before you type

With completely empty input, the same headless turn reported 31,979 tokens. That is the system prompt, the tool definitions, and the project instruction file, assembled before your prompt is considered at all.

Against the two window sizes Claude Code actually reported on this machine on 2026-08-18 — 200,000 tokens for Claude Haiku 4.5 and 1,000,000 for Claude Sonnet 5 and Claude Opus 5 — that floor is:

Sixteen percent of the smaller window is spent before the session does anything. It is not waste — tool definitions are what make the agent an agent — but it is not visible either, and it is the reason a "small" task can start closer to its token limit than it looks. We later split that floor in two by emptying the tool list: the tool definitions alone were 22,953 tokens, which is 71.9% of it.

Your own instruction file rides on top of that. Ours is 8,137 bytes, which the measured ratio prices at 2,393 tokens, and it is loaded in full on every request regardless of relevance. That is the mechanism behind keeping CLAUDE.md short: the file is not read when needed, it is paid for every turn.

One lockfile, 139,640 tokens

The single most useful number in the table is the one nobody plans for. package-lock.json in this repository is 309,043 bytes, and it measured 139,640 tokens — 69.8% of a 200,000-token window, consumed by a generated file that no human reads.

We then pushed it further, deliberately, to find out what an ordinary agentic read actually costs. Three committed files — the lockfile, the roadmap, and the editorial standards document, 696,820 bytes in total:

cat package-lock.json roadmap.md content-rules.md | claude -p "Reply with exactly: ok" --output-format json --effort low --model sonnet
# → 354,835 context tokens, total_cost_usd 1.960971

354,835 tokens and $1.96, for a turn whose entire output was the word "ok." Nothing about that read was exotic: it is three files from one small repository, and an agent asked to "understand this project" would plausibly open all three.

What a large context window in AI coding buys

A large context coding model removes exactly one failure mode: it lets a big read complete without compaction. On the 354,835-token turn above, a 200,000-token window would have failed and a 1,000,000-token window did not. That is a real difference and it is the whole of the difference.

What it does not buy:

• Accuracy over the window. Fitting the repository in is not the same as reasoning over all of it. Nothing measured here says anything about retrieval quality at 350k tokens, and we are not going to pretend otherwise.
• A lower bill. The same turn cost $1.96. A window ten times larger is an invitation to spend ten times more, and the pricing is linear where the value is not.
• Freedom from the floor. The 31,979-token floor is charged on every request in both window sizes.

The decision rule is narrow, and it is a decision: pick the larger window when your work genuinely spans more files than fit in the smaller one, and pick the cheaper model otherwise. For which Claude model that means in practice, the Sonnet and Opus comparison puts a ratio on it.

How often the larger window is actually load-bearing is measurable rather than hypothetical: across 20,324 turns on this machine, 46.9% carried more than a 200,000-token window holds, and the median turn sat at 96% of one. Those figures, and the fact that context never came back down in any of 89 sessions, are in what happens to the window once it is filling.

The measurement bug that cost 19 percent

The first run of this experiment was wrong, and the way it was wrong is worth more than the table.

The baseline was measured first, on a cold session. A session's first turn writes the prompt cache rather than reading it, so it reports a smaller cache_read and a larger cache_creation than every turn after it — 21,858 against 26,126 on the run that caught it. Subtracting a cold baseline from warm runs moved README.md from 3.399 to 4.062 bytes per token, a 19% error, and it landed almost exactly on the folklore constant the experiment existed to test.

That is the dangerous kind of wrong: a broken method that agrees with the thing everyone already believes.

The fix is one discarded turn before the baseline, and it is in the script with the reason written next to it. The guard is separate: the script asserts that every file run reports the same cache_read and refuses to print a table when they disagree.

Best practices for managing the window

• Measure your own repository, don't estimate it. Run npm run check:fill against your files. The ratios above are ours; a Python monorepo or a Go service will tokenise differently.
• Find your biggest generated file before the agent does. Sort by bytes, and check whether a lockfile, a snapshot, or a build manifest could be read on a whim. Ours is 69.8% of a small window.
• Treat the instruction file as a per-request cost, not storage. It loads in full every turn. Every line you add is a line you buy again on the next request.
• Budget from the floor up. Start at 31,979 tokens spent, not zero, and size the work against what is left.
• Pick the window from the job, not the spec sheet. A large window is worth paying for when a real read does not fit — and that is a question you can answer by measuring, in about a minute.

Common mistakes with token limits

• Dividing bytes by four. The mistake is tempting because the number is famous and roughly right for prose. The symptom is a session that compacts earlier than your budget said it would. The fix is a measured ratio per language, and a check that fails when it drifts.
• Benchmarking on a cold session. Tempting because it is the natural order to run things in. The symptom is a result that quietly confirms whatever you expected. The fix is a discarded warm-up turn, and an assertion on cache_read so a cold run cannot pass silently.
• Reading the whole repository to "give it context." Tempting because the window is big enough now. The symptom is a $1.96 bill for the word "ok". The fix is opening files on demand, which is what the tool's search tools are for.
• Assuming the window is the limit that will bite you. It usually is not. On the measured turns the output ceiling was 64,000 tokens against a 1,000,000-token input window, so the thing you run out of first is generally room to write, not room to read.

What we are not claiming

These ratios are Anthropic's tokeniser, as Claude Code 2.1.234 reported it on 2026-08-18. They are not portable: OpenAI and Google tokenise differently, publish differently, and their figures are not interchangeable with these — for the general version of that problem, what the price cards actually say is the article. The size of that gap has since been measured on these same files: the eight of them that have not changed are 1.206x more tokens on Anthropic's tokeniser than on OpenAI's, which is what pricing per byte rather than per token exists to correct.

The method measures stdin, not a tool call. A Read tool result carries call framing and a result envelope on top of the file's bytes, so a file opened by the agent costs slightly more than the same file piped in. We did not measure that overhead and are not going to guess at it.

And nothing here is a quality claim. This measures what a file costs, not what the model does with it.

Conclusion

Measure your own files before you trust any token estimate, including this one. Run npm run check:fill or the two-turn subtraction behind it, find the largest generated file in your repository, and start your budget at the floor rather than at zero — on this machine that floor was 31,979 tokens and the four-characters-per-token shortcut was wrong about every file we tried. If you are choosing a model off the back of it, the window size is worth paying for only when a read you actually need does not fit in the smaller one.

Frequently asked questions

Muhammad Kashif

Founder and editor of Devventa, covering AI coding assistants, Next.js and the modern AI development stack.