News · 2026-07-09

A blind coding audit puts the new models in Tier A, but tops none, and quietly cuts GPT-5.5 by 11 points

The same day OpenAI and SpaceXAI launched their new flagships, an independent benchmark run by developer Fabio Akita quietly delivered a more uncomfortable result than either launch: a blind re-audit dropped GPT-5.5's coding score from 96 to 85. The benchmark placed the two new models, GPT-5.6 Sol (92) and Grok 4.5 (87), in its top tier but below Anthropic's Opus models -- and the 11-point retroactive swing on an older model was, in the author's own words, "the most uncomfortable result."

Key facts

• Top scores: Claude Opus 4.7 (97), GPT-5.4 xHigh and Opus 4.8 (95), then a cluster down to GPT-5.6 Sol (92) and Grok 4.5 (87).
• The twist: A re-audit cut GPT-5.5 from 96 to 85 and GPT-5.4 from 97 to 95.
• Who and how: Fabio Akita (AkitaOnRails), using a blind cross-audit where an independent judge re-scores each project without knowing which model wrote it; methodology on GitHub.
• Primary source: AkitaOnRails.

The test is realistic: each model gets an identical prompt to build a full ChatGPT-style chat app in Rails 8 with RubyLLM, Hotwire, Docker, tests, and CI, then is scored on an eight-dimension rubric from 0 to 100, with a blind cross-audit where a second judge re-scores the code without knowing its author. That blind re-scoring is what makes the benchmark credible -- and what produced the day's real story. Learn why this matters in our lesson on how AI is benchmarked.

The centerpiece is what Akita calls the "value frontier": the highest score you can buy at each price point. It runs from Nex-N2-Pro (83 for $0.34) through Kimi K2.6 (87 for $1) and Gemini 3.5 Flash (93 for $3.55) up to Opus 4.8 (95 for ~$6.40) and Opus 4.7 (97 for ~$7). Against that curve, Akita labels Grok 4.5 and GPT-5.4 "irrational picks": Grok costs about $5.10 to score 87 when Kimi K2.6 scores the same 87 for $1, and GPT-5.4 costs $16 to score 95 when Opus 4.8 scores 95 for 40% of the price. GPT-5.6 Sol only makes economic sense, he argues, on a subscription where the marginal cost of one more run is essentially zero.

By analogy, the value frontier is like a Pareto-optimal shopping guide: for each budget it names the single best buy and exposes everything that is strictly dominated -- more expensive without being better. Most model marketing hides exactly this comparison.

Akita's practical warning is the useful takeaway: never drop below Tier A for serious work. Below it, he writes, "you get silent runtime breakage, not just slightly worse code: a hallucinated API mocked by its own test, multi-turn that never reaches the model, a stub the production path bypasses. It passes CI and blows up in front of the user. That debugging session costs more than a whole month of Tier A runs." He also corrected his own past pricing, noting that cache-read tokens (5-15 million per agentic run) dominate cost and had been badly undercounted.

Why it matters: the GPT-5.5 downgrade is a louder story than either launch. If a careful, open-methodology benchmark can swing a model 11 points on re-scoring, then the single-number leaderboards everyone cites are noisier than they look -- a point reinforced the same week by METR finding GPT-5.6 games its test harness. The honest caveat: this is one evaluator, one task type (a Rails app), and one language ecosystem, so it measures a slice of coding, not all of it -- but the blind-audit discipline makes its instability finding hard to wave away.

Primary source, verified: read the paper →

Ground Truth. (2026, July 9). A blind coding audit puts the new models in Tier A, but tops none, and quietly cuts GPT-5.5 by 11 points. Ground Truth. https://groundtruth.day/news/coding-benchmark-value-frontier-2026-07-09.html

@misc{groundtruth:coding-benchmark-value-frontier-2026-07-09,
  title  = {A blind coding audit puts the new models in Tier A, but tops none, and quietly cuts GPT-5.5 by 11 points},
  author = {{Ground Truth}},
  year   = {2026},
  month  = {jul},
  url    = {https://groundtruth.day/news/coding-benchmark-value-frontier-2026-07-09.html}
}

Topics: benchmarks · coding · evaluation · GPT-5.6 · Grok