News · 2026-08-23

Ornith-1.5 writes its own training problems and grades them

Ornith has released Ornith-1.5, an open-weight model family whose training loop invents its own problems. Rather than learning from a fixed set of human-written tasks, the system proposes progressively harder ones based on what it has already solved, builds the scoring harness for each, attempts a solution, and pushes the resulting reward back through all three stages at once. The 397-billion-parameter flagship scores 86.1 on Terminal-Bench 2.1, effectively level with Claude Opus 4.8's 85.0.

Key facts

• Three sizes: 397B and 35B mixture-of-experts models plus a 9B dense model, with a quantized 9B Mobile variant for phones.
• Flagship results: 86.1 on Terminal-Bench 2.1 and 56.0 on DeepSWE, against 85.0 and 59.0 for Claude Opus 4.8.
• The task generator targets roughly a 20 percent success rate, so problems stay just past what the model can currently do.
• Primary source: Ornith's release post, with weights in the Hugging Face collection.

Reinforcement learning on verifiable tasks has become the standard way to make a model better at coding and agentic work, and it has a well-known bottleneck: someone has to write the tasks. Human-curated problem sets are expensive, they run out, and once a model solves most of them there is nothing left to learn from. The usual patch is synthetic data -- have a model write more problems. Ornith's argument is that generating problems is not enough on its own, because a model asked to write practice questions will happily write easy ones, malformed ones, or near-duplicates of what it already knows.

Ornith-1.5's answer is to make task generation a rewarded behavior in the same loop as solving. As the release describes it, "Reward from the rollout is propagated across all three stages, so the system learns not only to produce better solutions, but also to generate more useful training tasks and construct more effective scaffolds." The task reward is a product of three terms, and the multiplication is the safeguard: a task must be valid, meaning it and its grader actually run and reliably separate correct from incorrect answers; it must sit near the model's frontier, judged from the empirical success rate across sampled attempts with a target around 20 percent; and it must be novel against a buffer of tasks already generated or trained on. Score zero on any one and the whole reward collapses, so a proposer cannot farm reward by writing impossible problems or restating solved ones.

The self-correcting part is elegant. Because difficulty is measured from the current model's own attempts, the curriculum moves on its own -- as the model gets reliable at a task, the reward for generating that task falls, pushing the proposer toward harder material. It is the difference between a fixed textbook and a tutor who watches what you get wrong and writes tomorrow's problem set from it.

The harness gets scored too, on whether it faithfully reflects the task, whether its rewards track real solution quality, and explicitly on its resistance to shortcuts and reward hacking -- a real risk when the thing being graded and the grader come from the same model.

The family spans an unusually wide deployment range for one release. The 35B model activates about 3 billion parameters per token, a mixture-of-experts design meant for two 80 GB GPUs at 256K context, and Ornith reports it beating same-size peers and considerably larger dense models on agentic coding by wide margins. The 9B dense model is the edge story: a 262,144-token window, OpenAI-compatible tool calling, and a quantized Mobile build the company says runs on phones.

Why it matters: if a model can reliably manufacture its own frontier curriculum, the supply of training tasks stops being the limiting reagent for agentic capability -- and that limit has been the quiet reason progress on coding agents depends so heavily on a handful of curated benchmark suites.

The caveats are real. Ornith-1.5 is built on top of Ornith-1.0, which the company says was itself developed from Qwen3.5 and Gemma 4 with additional continued pretraining -- so this is a post-training method layered on other labs' base models, not a from-scratch family. The benchmark comparisons are the vendor's own and the peer set is self-selected. And the deepest problem with self-generated evaluation is one no release post can settle: the same model writes the exam and takes it, so an error in its idea of correctness propagates into the reward signal invisibly. The published anti-hacking terms are a mitigation, not a proof. It is worth remembering how often coding agents ace the public test and stumble on the hidden one.

Primary source, verified: read the paper →

Ground Truth. (2026, August 23). Ornith-1.5 writes its own training problems and grades them. Ground Truth. https://groundtruth.day/news/ornith-1-5-writes-its-own-training-problems-and-grades-them.html

@misc{groundtruth:ornith-1-5-writes-its-own-training-problems-and-grades-them,
  title  = {Ornith-1.5 writes its own training problems and grades them},
  author = {{Ground Truth}},
  year   = {2026},
  month  = {aug},
  url    = {https://groundtruth.day/news/ornith-1-5-writes-its-own-training-problems-and-grades-them.html}
}

Topics: open-weights · reinforcement-learning · coding-agents · curriculum-learning · synthetic-data