News · 2026-11-11

Fable and Carbon built six apps; both produced serious failures

A six-project coding comparison scored Anthropic’s Fable 5.2 setup at 42 out of 60 and Google’s Gemini 4 Carbon setup at 32, while finding serious failures in software produced by both. The test compared complete coding-agent systems, so its result cannot isolate the model from the tools, resource budgets and review methods around it.

Key facts

• One creator gave the systems six matching app-building prompts.
• Fable ran through Claude Code; Carbon ran through OpenCode and a different provider route.
• Both container projects failed hostile-image security tests in the reviewer’s account.
• Primary source: AICodeKing’s six-app comparison; product documentation from Anthropic and Google.

Anthropic positions Fable for ambitious technical projects and recommends another model for much routine work. Google describes Carbon as a higher-effort model for longer tasks. Both are systems that can reason through tasks and use tools when a surrounding application gives them access.

AICodeKing describes using “the same six app builds” and the same prompts in the comparison. The reviewer asked for an emulator, container runtime, room designer, keyboard application, game and health application. Matching the prompts is useful, but the comparison is more like testing two racing teams than two engines on the same test stand: tools, pit strategy and time spent all affect the result. Both were set to “High,” a label whose meaning is provider-specific.

The failures are more instructive than the overall score. Both container builds allowed escapes in the reviewer’s hostile tests. The Fable health application stored access tokens in an unencrypted local database; Carbon substituted plausible values for missing activity and sleep data. The reviewer also found a room designer control that did not work despite the agent’s own passing tests. These are reported observations, not independently rerun artifacts.

The creator estimated about $222 for Fable and $23 for Carbon, but Carbon’s route reported almost no reasoning-token usage. Google’s pricing says those tokens are billed as output. Pricing also changes after the introductory window, so the two estimates cannot establish a general cost ratio.

The strongest counterweight to a simple Fable victory is Artificial Analysis’s High-effort comparison. It reports an index score of 62 versus 60, with Carbon ahead on some automation and terminal tests. Its benchmark has its own task mix and setup. The site's lesson on paired comparisons explains why shared tasks help without automatically controlling every difference.

ARC Prize’s Fable result makes the wrapper issue concrete: High-effort performance varied from 79.3% with its standard harness to 97.7% with an adapter preserving additional state. Neither a visually convincing demo nor a headline score proves a generated application is secure. A practical evaluation needs realistic use, adversarial checks and complete accounting.

What the price and benchmark actually measure

Google’s introductory Carbon rate is $2 per million input tokens and $10 per million output tokens through November 30, with published standard rates of $4 and $20 from December 1. Anthropic lists $8 and $40 for Fable’s ordinary input and output rates. Those are unit prices. A task’s total cost also depends on cached input, output volume, tool calls, retries and how much reasoning the system actually performs. A lower price per token does not guarantee a lower final bill.

Artificial Analysis reports that Carbon used substantially more output tokens than Fable across its index and took longer per index task in that run. It nevertheless reported a lower average cost per index task for Carbon under the prices it used. Those numbers belong to that evaluator’s tasks and configuration. The evaluator’s methodology explains the construction of its composite; the score does not promise the same ordering on a particular company’s codebase.

Anthropic’s own benchmark notes include another warning about effort. More computation may improve some tasks while costing more, and the relationship is not guaranteed to improve monotonically. Google’s model card compares with earlier competitor versions and discloses variations in evaluation settings, including a subset and best-of-three treatment for one computer-use test. A vendor’s comparison table is useful for understanding its published tests, but should not be silently treated as a fresh, symmetric trial of both latest models.

In the app test, neither keyboard application was checked on a physical keyboard. The reported security and interaction findings therefore coexist with gaps in the review. The strongest next comparison would publish task artifacts, fixed resource limits, settings and invoices, then ask independent reviewers to run the same security and usability tests. A better average would still be only one input into a deployment decision.

Primary source, verified: read the paper →

Ground Truth. (2026, November 11). Fable and Carbon built six apps; both produced serious failures. Ground Truth. https://groundtruth.day/news/fable-carbon-coding-apps-comparison.html

@misc{groundtruth:fable-carbon-coding-apps-comparison,
  title  = {Fable and Carbon built six apps; both produced serious failures},
  author = {{Ground Truth}},
  year   = {2026},
  month  = {nov},
  url    = {https://groundtruth.day/news/fable-carbon-coding-apps-comparison.html}
}

Topics: coding-agents · models · benchmarks