How AI Coding Agents Work: Model Plus Harness

An AI coding agent is a model plus a harness, and the harness is the larger half. On 2026-08-19 we ran one question about this repository through Claude Code 2.1.235 twice, changing nothing but the tool list: with the harness the model answered 55, which is correct, and with the tool list emptied the same model answered 51. This article separates the two halves on a real machine, measures what each contributes, and shows the loop that turns a stateless text function into something that can open your files.

Key takeaways

• The model cannot see your repository. With every tool removed it guessed the file count wrong in all seven runs we made, returning 43, 48, 49, 51 and 52 against a true 55.
• Tool definitions were 71.9% of the loaded context — 22,953 tokens of the 31,935 the session carried before the question was read.
• One question is not one model call. A single-tool question took two round-trips; a three-part question took three round-trips and five tool calls.
• Every round-trip re-sends the whole conversation, because the model holds no state. The three-part task sent 99,747 input tokens for a transcript that ended at 34,133.
• The loop running is not the same as the answer being right. The three-part run gathered its evidence correctly and still named the second-largest file as the largest.

The short answer

How AI coding agents work: a stateless model proposes tool calls, a harness executes them on your machine, and the loop repeats until the model stops asking. Everything people attribute to "the agent" — reading files, running tests, editing code, stopping — is harness behaviour. The model contributes the decision about what to ask for next.

That split is why two tools running the identical model produce different results, which is the mechanism behind what actually separates Claude from GPT for coding. If you are choosing a tool rather than a model, the complete comparison of AI coding assistants is the pillar this sits under.

What a harness actually is

A harness is four things wrapped around a model that has none of them:

• A system prompt. Standing instructions the model receives before your message, every time.
• Tool definitions. A JSON schema per tool, describing its name, arguments and behaviour. This is how the model learns that a tool exists at all.
• An executor. The code that receives a tool_use block, runs it against your filesystem or shell, and appends the result to the conversation.
• A loop. Send, receive, execute, append, send again — until the model returns text instead of a tool call.

Claude Code publishes its own list at the start of a headless run. With MCP disabled it advertised 32 tools on 2026-08-19; the same list can be handed straight back to the CLI as a removal list, which is what makes the experiment below possible:

claude -p "How many .mdx files are under content/? Reply with only the number." \
  --output-format stream-json --verbose --strict-mcp-config --model haiku
# → init event carries: "tools": [ ...32 names... ]

The whole instrument is committed as scripts/check-agent-loop.mjs:

npm run check:loop

The same model with no harness at all

Pass all 32 advertised tools to --disallowedTools and the model gets one turn, no tool channel, and no way to look at anything. Same model, same question, same working directory:

There are 55 .mdx files under content/. The script counts them itself before it grades anything, so neither run is scored on how confident it sounds.

The bare model did not decline. It answered, fluently, and it was wrong every time. The two runs in the committed fixture returned 51 and 49; five earlier runs during development returned 43, 48, 49, 52 and 52. Seven attempts, not one of them 55.

The more interesting part is how it failed. Stripped of a tool channel, the model wrote the tool call into its prose instead — nine lines of markup that no executor was listening for:

<function_calls>
<name>Glob</name>
<parameter name="pattern">content/**/*.mdx</parameter>
</invoke>
</function_calls>
<function_calls>
<name>Glob</parameter>
<parameter name="pattern">content/*.mdx</parameter>
</invoke>
</function_calls>

51

Look at the closing tags. <name>Glob</parameter> is not valid anything — the markup is improvised, because it is being written as prose rather than emitted through a serialiser. The disposition to reach for a file-listing tool is in the model. The channel that makes the reach mean something is in the harness, and with the channel gone the reach is just text that happens to contain angle brackets.

Tool definitions are most of what the model reads

The two configurations differ in exactly one thing, so the difference between their loaded contexts is attributable: 22,953 tokens of tool definitions, 71.9% of everything the session carried before the question was considered.

That figure reproduced at exactly 22,953 across three separate measurement runs on the same day, and the repeat probe moved the floor by zero tokens, which is the evidence that the subtraction is measuring the tool list and not noise.

Two consequences follow. The first is a budget one: this is the tool-shaped part of the session floor that what actually fills a context window measured at around 32,000 tokens, and it is paid on every round-trip, not once. The second is a design one — every server you attach through an MCP configuration appends its schemas to that block, so a tool you never call still costs you on every turn of every task.

The agent loop resends everything, every turn

The model is stateless. It cannot remember the tool result it received two seconds ago, so the harness sends the entire conversation again on the next call. That is visible turn by turn, because every assistant message carries its own usage block:

One question, one tool call, 65,080 input tokens across the wire for a transcript that ended at 33,145 — a 1.96x resend. Now the same instrument on a three-part question that cannot be answered from one call:

Five tool calls, three round-trips, 99,747 tokens sent for a transcript ending at 34,133: a 2.92x resend. The multiplier tracks the number of round-trips almost exactly, because the session floor dominates the transcript and gets re-sent whole every time. Round-trips are also what a real session is made of: counted across 87 transcripts on this repository, one human sentence buys a median of 30 tool calls and once bought 236.

Running the loop is not the same as being right

The three-part run gathered its evidence properly and still got a third of the answer wrong. Graded against the same local truth the script computes for itself:

The largest file under content/ is best-ai-coding-assistant.mdx at 34,060 bytes. The agent had shell access, used it five times, reported a real byte count for a real file — and named the runner-up.

That failure is not a harness failure. The tools worked, the loop ran, the results came back. This is the half the model owns, and it is the reason a benchmark score is a claim about one of the two variables rather than about the agent. The script reports this grade and deliberately does not fail on it, because a check that treated a wrong answer as a broken fixture would delete the finding.

The experiment that gave a false negative

The first attempt at a no-harness run was hand-written. We listed the tools that seemed to matter — Bash, Glob, Read, Grep, Task and a few more — passed them to --disallowedTools, and ran it.

The model answered 55. Correct. For about a minute it looked as though the harness contributed nothing.

Two things had gone wrong, and both are visible in the stream:

• PowerShell was not on the hand-written list, so the harness still advertised it. On Windows that is a complete shell, which is to say the tool channel was never actually closed.
• The model emitted a Glob call anyway, for a tool that had been removed from its list, then fell back to PowerShell when that went nowhere.

The fix is in the committed script: the removal list is not written by a human, it is read back from the harness's own init event, so it is exactly the set of tools that run advertised. A guessed list is how you accidentally prove the opposite of what you measured, and it agreed with the comfortable answer — the same trap the context-window measurement fell into from the other direction.

Agent vs model as a decision rule

The distinction earns its keep when you are deciding where to spend. Use it like this:

• Wrong answers with correct evidence gathered are a model problem. The largest-file miss above is the shape: the tools ran, the data came back, the reading was wrong. A stronger model fixes this; a better prompt sometimes does.
• Right reasoning about the wrong files is a harness problem. If the agent never opened the file that mattered, no model upgrade helps. Change what the harness reads — that is what a project instruction file is for, and why keeping CLAUDE.md short is a performance decision.
• Cost is almost entirely a harness problem. Round-trips, resend, and the tool-definition block are all harness-side. The model's per-token rate is the multiplier, not the driver.
• Capability ceilings are harness ceilings first. An agent cannot use a tool it was not given. Before concluding a model "can't do X", check whether X was on the list — what the tool actually reports about its own model is a good habit for the same reason.
• Parallelism is a harness feature. Isolated sub-tasks with their own context are a subagent mechanism, not a model one.

Common mistakes about agent vs model

• Reading a benchmark score as an agent score. Tempting because the leaderboard names the model. The symptom is a model that scores well and disappoints in your editor. The fix is to check which harness produced the score — what a SWE-bench number actually measures covers the rest of that gap.
• Adding MCP servers for capability without counting them. Tempting because each one is individually small. The symptom is a session that compacts earlier than it used to for no visible reason. The fix is to measure the tool block; here it was 71.9% of the floor with no MCP servers attached at all.
• Assuming a tool result is free after it arrives. Tempting because you only paid for the read once. The symptom is the 2.92x figure above. The fix is fewer, larger round-trips — one command that answers three questions rather than three commands.
• Testing "can the model do this" without the harness. We made this one. The symptom is a result that flatters whichever side you forgot to control. The fix is to derive the configuration from the tool rather than writing it by hand.

What we are not claiming

This is one harness. Claude Code is the only agent installed on this machine, so every number here describes how a harness is built, not which harness is best. scripts/check-harness-share.mjs on this site exists to check the preconditions for a cross-tool comparison, and it fails here on purpose.

It is also one tokeniser and one model family. The 22,953-token tool block is Anthropic's schema serialisation as Claude Code 2.1.235 emitted it; another vendor's tool definitions for the same 32 capabilities would not weigh the same.

And the bare-model runs are not a claim that models are bad at counting. They are a claim that a model with no tool channel cannot observe a filesystem, which it cannot, and the seven wrong answers are what "cannot" looks like when the model answers anyway.

Conclusion

Treat "agent" and "model" as separate purchases, because they fail separately. Run npm run check:loop or the two-configuration probe behind it against your own setup: empty the tool list and you will see what the model alone knows about your repository, which on this machine was nothing it could get right. Then count the tool definitions before you add another server, and count round-trips before you blame the model for the bill — on a three-part question the loop sent the conversation nearly three times over, and the model's only mistake was reading one number wrong.

Frequently asked questions

Muhammad Kashif

Founder and editor of Devventa, covering AI coding assistants, Next.js and the modern AI development stack.