Best AI Model for Python: 46% of the Evidence Is Django

The best AI model for Python is the only "best model for a language" question the published evidence can actually answer, and it answers it more narrowly than the leaderboards suggest. We downloaded all 500 instances of SWE-bench Verified on 2026-08-19 and counted every file its gold patches touch: 622 of 623 are .py, all twelve source repositories are Python projects, and 231 of the 500 instances — 46.2% — come from Django alone. This article shows the count, what it supports, and what Python's own tokenisation adds to the decision.

Key takeaways

• SWE-bench is a Python benchmark by construction, not by reputation: 99.84% of the files its solutions edit carry a .py extension, and the one exception is a .cfg configuration file.
• Django supplies 46.2% of SWE-bench Verified. With SymPy at 15.0%, two codebases account for 61.2% of the evidence behind every headline coding score.
• A model's coding score generalises to Python and does not obviously generalise anywhere else — SWE-bench Multilingual, the natural contrast, returned HTTP 401 to our reader on 2026-08-19.
• Python holds about 1.10x more source per token than TypeScript — 3.693 bytes per token against 3.353, measured on real CPython, Django and SymPy files.
• One Python file measured 4.028 bytes per token, breaking a boundary this site published two days earlier off nine files from one repository.

The short answer

For Python, trust the leaderboard more than you would for any other language, and then discount it for distance from Django. The number is measured on Python and only on Python, which makes it real evidence — and nearly half of it comes from one mature web framework's issue history, which caps how far it travels.

If you are picking a tool rather than a model, the complete comparison of AI coding assistants is the pillar this sits under, and what a SWE-bench number actually measures covers the construction pipeline this article counts the output of.

Counting the benchmark instead of quoting it

Every explanation of SWE-bench's language coverage traces back to one sentence in the original paper. The dataset is public, so the sentence can be checked. scripts/check-python-evidence.mjs pulls all 500 rows from the Hugging Face datasets server, extracts every +++ b/<path> line from every gold patch, and counts extensions:

npm run check:python

What the solution patches change, across all 500 instances:

The test patches are marginally less uniform — 649 .py, seven .rst, four .txt and one .json — and the .rst files are Sphinx doctests, which is to say still Python, executed from documentation.

There is no second language in this benchmark. Not a small share, not a long tail: one configuration file across 623 edits. Any claim that a SWE-bench score predicts performance on Go, Rust or TypeScript is an extrapolation with nothing under it, which is the same gap what the price cards say runs into from the cost side.

Django is 46.2 percent of the evidence

The repository distribution is the part no leaderboard prints, and it is severely lopsided:

The remaining seven — astropy, xarray, pytest, pylint, requests, seaborn and flask — share 17.8% between them. Flask contributes exactly one instance. Seaborn contributes two.

So a headline score is, in weighted terms, a measurement of how well a model resolves issues in Django and SymPy. Both are large, old, heavily-tested, heavily-documented Python codebases with strict contribution conventions — which is precisely why SWE-bench selected them, and precisely why the score is a poor proxy for a two-year-old service with sparse tests. The construction filter that produces this selection is documented in the SWE-bench explainer, which ran the same filter over this repository and got zero instances out of it.

The practical form of the discount: the closer your Python looks to Django, the more the leaderboard is telling you.

Python costs less context than TypeScript

There is a second, language-specific consideration that has nothing to do with benchmarks: how much source fits in a window. Measured through Anthropic's tokeniser on 2026-08-19, by differencing two headless turns so the session floor cancels:

Averaged by language, Python comes to 3.693 bytes per token against 3.353 for this repository's TypeScript and TSX — 1.10x more source per token. The mechanism is unsurprising once stated: Python spends no tokens on braces, semicolons or type annotations, and its keywords are ordinary English words that tokenise well.

Ten percent is not a reason to change language, and it is a real effect on a large read. On a 200,000-token window that is about 68,000 more bytes of source before anything is dropped — two-thirds of django/db/models/query.py, which is 105,536 bytes. For the general version of this budget, what actually fills a context window has the method and the session floor.

The file that broke a rule this site published

Two days before this article, our context-window measurement reported that every one of nine files came in under 4.0 bytes per token, and treated the four-characters-per-token constant as an upper bound that source code never reaches.

cpython Lib/argparse.py measures 4.028.

The original claim was scoped to nine files from one repository and remains true of them. The general reading it invited — that source code is always under four — does not survive contact with a wider sample. Long-form Python with extensive docstrings and prose comments tokenises like the English it largely is, and argparse is one of the most heavily documented modules in the standard library.

The correct statement is narrower: the ratio is a property of the file, not of source code. JSON measured 2.213 in that earlier sample and Python measures 4.028 here — a 1.82x spread across the same tokeniser. The earlier article now carries a dated note pointing here, because a correction to the standards and a correction to the corpus are two separate jobs.

What a Python score actually supports

Three claims, in decreasing order of how much the count supports them:

• "This model is good at resolving issues in mature, well-tested Python libraries." Directly supported. That is the population sampled.
• "This model is good at Python." Broadly supported, with the Django weighting as the caveat. Nothing in the benchmark exercises async-heavy services, data pipelines, notebooks or anything under 100 contributors.
• "This model is good at coding." Not supported by this benchmark at any strength. The evidence contains one non-Python file, and it is a config file.

That last gap is why the three-way price comparison on this site ranks cost and refuses to rank quality: a quality ranking needs evidence in the language you actually write, and for most languages it does not exist in public.

How to pick a model for Python

• Use the leaderboard, and weight it by distance from Django. If your codebase is a mature, well-tested library, the score is unusually good evidence. If it is a thin service with three tests, halve it in your head.
• Check the score is on Verified, not the full set. SWE-bench Verified is 500 human-checked instances; the unfiltered dataset is a different and much larger population, and the two are quoted interchangeably.
• Budget context in Python terms. At 3.693 bytes per token you fit about 10% more than the TypeScript figures most guides are written from — measure your own repository rather than inheriting either number.
• Decide the harness before the model. For any language, the tool that reads your files decides more than the model does — the model-and-harness split puts numbers on that, and it applies to Python exactly as it applies to everything else.
• Re-price the model separately from ranking it. The cheapest and the highest-scoring are rarely the same row; the Sonnet and Opus ratio is the clearest example of a decision that is entirely about cost.

Common mistakes picking a model for Python

• Reading a coding score as language-agnostic. Tempting because the leaderboard says "coding". The symptom is a model that impresses on a Django-shaped task and disappoints on your Rust service. The fix is to check the dataset's composition, which takes one script.
• Assuming the twelve repositories are equally weighted. Tempting because "12 repositories" sounds like a broad sample. The symptom is over-trusting a score on framework-free code. The fix is the distribution above: two repositories are 61.2% of it.
• Estimating Python token counts from a general constant. We published a version of this mistake ourselves. The symptom is a context budget that runs out early on JSON and late on docstrings. The fix is a per-language measurement.
• Treating an absent benchmark as an equivalent one. Tempting when a vendor cites a multilingual variant. The symptom is a citation nobody can check — ours returned 401. The fix is to say so, which is what this article does rather than quoting a figure it could not read.

What we are not claiming

No model is ranked here. This article counts what the evidence is made of; it does not run a single model against a single instance, and nothing in it says which model writes better Python.

One benchmark. SWE-bench Verified is the one most launches quote and the one whose rows are public. SWE-bench Multilingual would be the natural contrast and its repository returned HTTP 401 on 2026-08-19, so its composition is named as unknown rather than estimated.

The token ratios are Anthropic's tokeniser, as Claude Code 2.1.235 reported it, on six Python files from three projects. OpenAI and Google tokenise differently and their figures are not interchangeable with these. Six files is enough to establish that Python sits above this repository's TypeScript and not enough to publish a constant, which is why the script prints the individual measurements and not just the mean.

Conclusion

For Python, use the leaderboard — it is measuring your language, which is more than developers in any other language can say — and then discount it for how far your code sits from Django and SymPy. Run npm run check:python to see the count for yourself; it takes five requests and no API key. If you write anything else, treat a SWE-bench figure as evidence about Python that somebody has extrapolated, and go looking for the language-specific number before you spend on the strength of it.

Frequently asked questions

Muhammad Kashif

Founder and editor of Devventa, covering AI coding assistants, Next.js and the modern AI development stack.