Claude Code vs ChatGPT Coding: One Needed a Flag

Claude Code and ChatGPT both answered the same committed coding question correctly, twice each, on one laptop on 2026-09-21. The difference that mattered was not the answer — it was that only one of them would start. ChatGPT's agent at a terminal is Codex CLI, and at its own default configuration it failed with an HTTP 400 telling us to upgrade, three times in a row, because the config file it shares with the ChatGPT desktop app names a model that CLI version cannot use. One flag fixed it. Everything here is re-derived by npm run check:chatgpt.

Key takeaways

• Both agents answered correctly on all four arms. Two arms each, same task, same directory.
• Codex CLI would not start at its defaults — a 400 saying the configured model "requires a newer version of Codex", reproduced three times.
• One config directory has two writers. A single codex exec rewrites the shared model catalogue from 7 models to 6, and the 6 do not include the one the config selects.
• The two agents work differently: Claude Code made four typed Read calls; Codex ran two shell commands and the first one failed on both arms.
• Only one of them tells you what a turn cost. Codex reports tokens and no price at all.

The short answer

Compare the harnesses, not the models. On a small, well-specified task both vendors' agents got the right answer, and everything that separated them was operational: whether the tool starts at its defaults, how it reaches your files, whether it recovers from a failed command, and whether it tells you what it spent.

The full AI coding assistants comparison is the pillar this sits under. If you want the model-level version of this question, Claude against GPT on coding is that article, and it found a catalogue with no GPT-5 on it. If you want the configuration-portability version, Claude Code against Codex CLI measured what each tool reads from your repository. This one is about whether the thing runs.

What ChatGPT actually is at a terminal

"ChatGPT versus Claude Code" is a comparison between a chat product and an agent, which is not a fair fight and not the one people mean. The fair version is the one both vendors ship: an agent that reads your repository and edits it (such as described in the Claude Code CLI overview), authenticated by the plan you already pay for. For ChatGPT that is Codex CLI. OpenAI now has another persistent agent surface in dot, whose responsibilities and connections live inside ChatGPT, but it is not the terminal harness measured here.

On this machine, both are installed and both are current enough to run:

That last row is not trivia. It is the mechanism behind everything in the next two sections.

It would not start at its own default

The first arm was supposed to be a warm-up. It is the shortest possible turn: one word in, one word out, no tools.

codex exec --json "Reply with only the word ok and nothing else."
# → item.completed  type=error
#   "Model metadata for `gpt-6-astra` not found. Defaulting to fallback metadata…"
# → error  status 400
#   "The 'gpt-6-astra' model requires a newer version of Codex.
#    Please upgrade to the latest app or CLI and try again."

Two independent refusals for one model name. The CLI cannot find local metadata for it, and then the server rejects it for this CLI version. The second does not depend on the first — it still happens when the local catalogue does contain the entry, which we checked.

Nobody typed gpt-6-astra. It is the model line in ~/.codex/config.toml, written there by the desktop app, and the CLI obeys it. Reproduced three times on 2026-09-21, and reproduced again on every online run of the check, which costs nothing because it fails before a token is billed.

The fix is one flag with a model name from the CLI's own catalogue (documented in OpenAI's models guide):

codex exec --model gpt-5.6-terra --json "Reply with only the word ok and nothing else."
# → item.completed  type=agent_message  text="ok"
# → turn.completed  input_tokens=15237  output_tokens=5

One file, two writers

The catalogue the CLI reads lives at ~/.codex/models_cache.json, and the reason the default configuration fails is visible in what happens to that file when you run the CLI at all.

# read the cache, run one turn, read it again
# → before:  client_version 0.155.0   7 models   229,180 B
# → after:   client_version 0.152.1   6 models   162,908 B

A single codex exec replaces the desktop app's catalogue with the CLI's own, and the CLI's is the one missing the model the config selects. The flip takes about two milliseconds and the CLI's version then holds — we sampled it every half second for 11.5 seconds and it did not move until the app ran again.

Three version numbers describe this one product on this one machine: the CLI on PATH is 0.152.1, the desktop app writing its config reports 0.155.0, and the CLI's own updater last looked on 2026-09-05 and still believes 0.153.4 is the latest. The config is shared; the understanding of what models exist is not.

Both agents answered the same question

The task is a committed fixture used by earlier measurements on this site: four small JavaScript modules, exactly one of which holds an off-by-one error, copied alone into an empty directory so no instruction file is discoverable above it. The prompt asks for the function name and nothing else. The answer is pageSlice.

One setup note, because it is a difference in itself. Codex refuses to run outside a git repository — "Not inside a trusted directory and --skip-git-repo-check was not specified." We ran git init and one commit rather than passing the flag: a real project is already a repository, so initialising one matches reality, while the flag would have been us reaching for an override to make a measurement work.

Four arms, four correct answers. That is a fact about this task, not evidence about either vendor's coding ability, and the fixture asserts it in those words. One question with one right answer cannot rank two agents, and this article does not.

Typed tools against a shell

The interesting part is not the answer, it is the route to it, and the two agents take different ones.

Claude Code made five typed tool calls — one Glob then four Reads, one per file — and ran no shell command at all. Six turns, no failures, both models identical in shape.

Codex went through PowerShell. Two commands per arm, and on both arms the first one was this:

powershell.exe -Command 'rg --files; rg -n "function|=>" src test tests'
# → exit 1
#   The term 'rg' is not recognized as the name of a cmdlet, function,
#   script file, or operable program.

ripgrep is not on this machine's Windows PATH, and we reproduced that in a bare PowerShell session with no agent involved, so it is the machine rather than anything Codex did. The agent recovered unprompted on the next command, reading all four files with Get-Content in one go, and got the right answer. But it burned a turn finding out, identically, twice.

That is the architectural difference in one line: one agent has file tools, the other has your shell. A shell is more powerful and inherits every gap in your environment. On a machine where rg happens to be installed, that first command would have succeeded and this paragraph would not exist — which is the point. The same agent behaves differently on two developers' laptops.

A smaller version of the same split shows up when you hand each agent a model name that does not exist:

Only one of them reports a bill

Both CLIs emit structured JSON for a completed turn. They do not report the same things.

Claude Code reported $0.2469 for the Opus arm and $0.1139 for the Sonnet arm (tracked in real-time per Anthropic's Claude Code cost documentation). Codex reported tokens and no price, which is consistent with a plan-based product where a turn does not have a price — but it means there is no cross-vendor cost comparison to make here, and the fixture licenses none.

One number inside the Claude Code side is worth keeping. The cheaper arm sent more prompt tokens than the dearer one — 131,751 on Sonnet against 98,964 on Opus for identical work — which is the same effect the six-arm billing measurement found when the tier premium came out at 2.19x rather than the 2.5x on the rate card.

Which one to use

• If you want it to work at its defaults today, on this machine that was Claude Code. That is a statement about one laptop's configuration, and it would flip the moment the CLI catches up with the app.
• If you are on a ChatGPT plan, pass --model explicitly. Do not let a shared config file choose for you, and read the CLI's own catalogue rather than a marketing page.
• If your environment is unusual, prefer the agent with typed file tools. A shell-first agent inherits your PATH, and ours cost a turn on a missing binary.
• If you need to know what a turn cost, only one of them will tell you. That is a real operational difference for anyone billing work.
• If you want the model comparison instead, this site's attempt at ranking coding models explains why it refuses to publish one.
• If this is your first week with either, what each one says when you get something wrong scores twelve first-day messages, including the one that names a bypass flag and never mentions Reinitialized existing Git repository in D:/Projects/Deventa/devventa/.git/.

What we did not measure

Which agent writes better code. Nothing here scores code. One question, one right answer, four correct arms — the fixture asserts that this is not capability evidence and the article is written so it cannot be read as one.

Cost between vendors. Codex reports no price for a turn, so there is nothing to compare against. The seat-price comparison prices the plans instead.

ChatGPT the web product. This compares two terminal agents. Pasting code into a chat window is a different workflow with different failure modes and none of them are measured here.

Whether the blocker is general. The 400 depends on this CLI version against this config, both of which change. That is why the check re-runs it live rather than quoting it.

Anything at scale. One task, one laptop, one account per vendor, one day. Four arms is four arms.

Common mistakes comparing these two

• Comparing a chat window to an agent. ChatGPT's answer to Claude Code is Codex CLI; anything else is comparing products that do different jobs.
• Reading "it failed to start" as a model failure. No model was involved in the 400. It was a config file and a version number.
• Reaching for a bypass flag when a tool refuses. We were offered one and used git init instead. If you widen permissions until a measurement works, you measured your configuration.
• Treating token counts as costs across vendors. One of these two does not price a turn at all.
• Generalising a single read of a file two programs write. We did exactly that in a first draft and it was wrong within one run.

Conclusion

For coding work at a terminal, pick on operations rather than on the model: both agents answered correctly here, and the only thing that separated them was that one started and one needed a flag. If you are on a ChatGPT plan, set --model yourself and expect the desktop app and the CLI to disagree about what exists; if you are on Claude Code, expect typed file tools rather than your shell, and a dollar figure on every turn. We re-check these figures with an internal test script — it re-reads both versions, re-runs the failing default for free, and verifies which half of this article is still true.

Frequently asked questions

Muhammad Kashif

Founder and editor of Devventa, covering AI coding assistants, Next.js and the modern AI development stack.