reinforcement-learning
Process reward models: grading the working, not just the answer Lesson
A process reward model scores each step of a model's reasoning rather than only the final answer, which makes it possible to catch a wrong turn even when the conclusion happens to come out right - and to tell a lucky guess apart from genuine reasoning.
Policy entropy: when reinforcement learning makes an AI less willing to try another good path Lesson
Policy entropy measures how spread out an AI’s action probabilities are; it matters because reward training can improve the most likely answer while quietly collapsing useful alternative solution paths.
Ornith-1.5 writes its own training problems and grades them News
Ornith released an open-weight model family whose training loop generates its own tasks, builds its own scoring harnesses, and feeds the reward back into all three stages -- with the 397-billion-parameter flagship matching Claude Opus 4.8 on agentic coding benchmarks.
Models can train each other without a single correct answer News
A method called Co-RL trains language models with no labels at all by rewarding each model for agreeing with a different model's majority vote, matching and sometimes beating the same recipe trained with ground-truth answers.
A frozen model can look like it taught itself, and most self-improvement results never checked News
A new audit ran a completely untrained control model through the same self-training pipeline as the real thing and found it appeared to both learn and forget, meaning most reported self-improvement gains are measurement artifacts unless the null was measured too.
Three papers landed the same day arguing you should build the world, not the model News
EnvHarness, FACET, and SPADE all took the top spots on Hugging Face's daily paper list with the same underlying move, shifting effort from making the agent smarter to manufacturing the environments the agent practices in, with FACET releasing 6,020 ready-made terminal tasks.
Frontier multimodal models still cannot build a 3D world, and a new benchmark says under 60 percent News
VibeWorlding tests whether multimodal agents can turn a plain request into an interactive 3D scene end to end, and finds that frontier models including GPT-5.5 and Qwen3.8-Max succeed on fewer than 60 percent of tasks.
Reinforcement learning changed only 1 to 3 percent of a model's tokens News
A token-level analysis found that reinforcement learning's entire benefit to reasoning models is concentrated at a tiny number of uncertain decision points, and a training-free method reproduced most of it for roughly a thousandth of the cost.
Compressed memory stretched a 7,000-token model to 1.75 million News
A framework that chunks long documents, compresses each chunk into memory blocks and gates which blocks the model reads extrapolated from 7,000 tokens of training context to 1.75 million at inference, with half the peak GPU memory of a leading baseline.
Curriculum learning: why the order of the examples matters Lesson
Training a model on easy examples before hard ones can make it learn faster and end up better than showing the same data in random order, which is why frontier labs now build ladders of increasingly difficult tasks.
Loss of plasticity: why a network that trains forever stops being able to learn Lesson
Keep training a neural network on a stream of new tasks and it does not just forget the old ones - it gradually loses the ability to learn new ones at all, until it performs worse than a freshly initialized network.
The data firms behind frontier AI sell judgment, not labels News
Mercor, Surge AI and AfterQuery have all converged on the same product line -- reinforcement-learning environments, scoring rubrics, expert demonstrations and human evaluations -- turning graded professional judgment into a commodity input for frontier models.
A 4B search agent matches 30B by grading its own failed attempts News
ABSeeker trains a 4-billion-parameter web-search agent on 8,500 examples by working backwards from the known answer to score each individual search step, letting useful steps inside failed runs earn credit -- and matches agents roughly seven times its size.
Credit assignment: figuring out which step deserved the reward Lesson
Credit assignment is the problem of working out which of an agent's many decisions actually caused the eventual outcome, and it is the central difficulty in training anything that acts over a long sequence of steps.
A task factory ran fifteen rounds and broke the model grading it News
A new paper builds harder and harder terminal tasks by recursively rewriting accepted ones, and across fifteen rounds a fixed frontier solver's success rate fell from 90 percent to 2.5 percent, with the authors reporting no ceiling in sight.
Self-Play: How a Model Gets Better by Competing With Itself Lesson
Self-play is training a model by having it compete or cooperate against copies of itself, so the opponent gets harder exactly as fast as the learner improves and the training data never runs out.
An RL Trainer That Invents Its Reward When the Judge Says Nothing News
The published code for SpyRL, a reinforcement learning method built on the promise of fully verifiable rewards, silently substitutes randomly generated votes with a hard-coded 60 percent accuracy rate whenever no judge outputs are present.
Imitation Learning: Teaching Machines by Showing, Not Rewarding Lesson
Imitation learning trains a system by having it copy demonstrations from an expert rather than discover behaviour through trial and error, and behaviour cloning - simply predicting what the expert did in each situation - is the simplest and most widely used version of it.
A $500 fine-tune of a 9B open model beat all five frontier models it was tested against News
A consultancy reinforcement-trained a 9-billion-parameter open model on a simulated product-catalog review workflow for about $500 of GPU time, and it outscored the best of five frontier configurations while costing $0.50 per thousand listings against $34.
Reinforcement Learning with Verifiable Rewards (RLVR) Lesson
RLVR trains a language model by rewarding answers that an automatic checker can confirm are correct - a passing unit test, a right final number, a valid proof - instead of using human ratings or a learned reward model. It is the training recipe behind the modern wave of reasoning models.
A trillion-parameter model taught itself to reason without ever seeing a human's worked solution News
Researchers scaled "zero RL" training to a trillion-parameter model, called Ring-Zero, and found the reasoning that emerges qualitatively changes at that size, reaching 84.2% on a hard math-competition exam without ever training on human chain-of-thought examples.
Markov Decision Processes: The Math Behind How AI Learns to Act Lesson
A Markov Decision Process is the mathematical framework that formalizes sequential decision-making -- states, actions, rewards, and transitions -- and it is the foundation on which nearly all reinforcement learning, from game-playing AI to robot control, is built.
Richard Sutton's Oak Lab bets against frozen models: a trillion-parameter agent on 20 watts News
Turing-winner Richard Sutton launched Oak Lab with a north-star goal of a trillion-parameter agent that learns and plans in real time on about 20 watts, betting on continual experiential learning over the static pre-train-then-freeze paradigm behind today's LLMs.
Sim-to-Real Transfer: Teaching Robots in a Dream Lesson
Sim-to-real transfer is the practice of training a robot's brain entirely in a fast, cheap computer simulation and then deploying it on real hardware, using tricks like domain randomization to survive the gap between the virtual world and the messy physical one.
'World model' was too vague, so researchers wrote a 58-page definition News
With 'world model' stretched to mean everything from video generators to robot simulators, a 58-page technical report set out to pin down a single scientific definition - as a wave of new papers pushed the field from world-models-as-pretty-video toward world-models-as-working-tools.
The RL 'mirage': the policy you optimize isn't the one you ship News
A top-ranked paper shows that in modern RL training, improving the model you optimize does not guarantee improving the model you actually deploy - because the two run on different engines that disagree on probabilities.
Group Relative Policy Optimization (GRPO) Lesson
GRPO is the reinforcement learning method behind most modern reasoning models: instead of training a separate 'critic' network to judge answers, it samples a whole group of answers to each question and scores each one by how it compares to the group's average.
Reward Hacking: When AI Games the Metric Instead of Doing the Job Lesson
Reward hacking is when an AI scores well on the objective you measured while defeating the outcome you actually wanted -- like a coding agent that passes every test by faking the result rather than building the product.
Three Popular Ways to Train Reasoning AIs Turn Out to Be One Formula News
A new proof shows that three widely used reinforcement-learning recipes for training reasoning models - GRPO, Dr. GRPO, and DAPO - are all just different operations on a single number, the spread of rewards within a group of sampled answers.
ByteDance says AI agents double their learning speed every three months News
ByteDance's Seed team released EdgeBench, a benchmark of 134 day-long tasks, and reported that agents' rate of learning from real environments has roughly doubled every three months -- a possible new scaling law measured over about 38,000 hours of agent activity.
On-Policy vs Off-Policy Learning Lesson
On-policy learning trains a model on data generated by its own current behavior, while off-policy learning trains it on data generated by something else -- an old version, a different policy, or a fixed dataset -- and the choice shapes how stable, sample-efficient, and reliable the training is.
Qwen used human-feedback training to make its image AI follow directions better News
A new recipe applies the same reinforcement-learning approach that polished chatbots to an image generator, then merges separate skill models into one - improving how faithfully it follows prompts and edits.
Put AI agents in charge of a Civilization game and they reach for the nukes News
A new benchmark let language-model agents play Civilization VI -- and they learned that the fastest path to winning ran straight through mutually assured destruction.
A wave of new methods trains AI without a human answer key News
Several research groups landed on the same idea at once - improve a model by learning from its own attempts instead of expensive human labels - and the field is debating whether it really removes the labeling burden or just hides it.
Why teaching AI agents to use tools keeps blowing up in training News
A new paper pins the sudden collapse of multi-step tool-use training on runaway probabilities in a few control tokens, and shows that mixing in supervised examples stabilizes it.
Alibaba's new models let AI agents practice in a world they imagine News
Qwen-AgentWorld trains a model to simulate the environment an agent acts in, then uses that simulation as a cheap, controllable place to learn -- reporting gains beyond training in the real thing.
AI Agents Are Learning to Build the Worlds They Train In News
Three new open research projects point the same way: instead of only learning what to do, agents are learning to simulate the environment itself, so they can practice in their own imagination.
What are world models? Lesson
A world model is an AI system's internal understanding of how an environment works — not just what it sees right now, but what will happen after an action, and what would have happened differently. Central to planning, robotics, and the next generation of physical AI.
slime Tool
The open-source large-scale asynchronous training framework from THUDM that Z.ai used to run the post-training scaling behind GLM-5.3.
prime-rl Tool
Open-source RL post-training stack that splits rollout generation and gradient updates across GPUs, used in this week's widely discussed $500 fine-tune that beat five frontier configurations on a catalog-review workflow. Practical for teams that already have an automatically scored task and want to train a specialist rather than pay per call for a frontier model.
VibeWorlding-Gym Tool
A Blender-backed sandbox that exposes 3D asset retrieval, editing and rendering as Model Context Protocol tools, plus a rubric verifier scoring physical feasibility and intent fulfilment. Usable as a training environment or as a plain MCP toolchain for 3D agents.
Qwen-AgentWorld Tool
Alibaba's open language world model that simulates agent environments -- browser, terminal, phone, coding workspace and more -- so other agents can be trained inside the simulation. Released with open weights and code in two sizes.
FACET terminal-agent task set Tool
A public release of 6,020 synthesized terminal-agent tasks plus three fine-tuned checkpoints. Each task bundles an instruction, an initialized environment, a reference solution, and an executable verifier, all grounded in the same container state so they cannot drift apart. Directly usable as reinforcement-learning environments for coding and shell agents.
ABSeeker Tool
A released 4-billion-parameter web-research agent trained with per-step credit assignment that matches roughly 30-billion-parameter agents on hard fact-finding tasks.