rl-post-training
Reinforcement Learning with Verifiable Rewards (RLVR) Lesson
RLVR trains a language model by rewarding answers that an automatic checker can confirm are correct - a passing unit test, a right final number, a valid proof - instead of using human ratings or a learned reward model. It is the training recipe behind the modern wave of reasoning models.
LongStraw Makes Million-Token RL Training Possible on 8 GPUs - But Its Code Doesn't Run Yet News
A new systems paper called LongStraw shows reinforcement-learning post-training can execute on prompts beyond 2 million tokens on a fixed 8-GPU budget by scoring the shared prompt once without gradients and backpropagating only through the short generated response, though the authors call this proof of execution capacity rather than full training correctness, and the public code is not yet runnable.
A weaker model just taught a stronger one, by passing on the lesson instead of the answers News
Researchers from Tsinghua and ByteDance show that a small model's reinforcement learning gains can be distilled into a larger, already-stronger model by transferring the change in the teacher's policy rather than its outputs, letting a 1.5B teacher improve a 7B student.
The RL 'mirage': the policy you optimize isn't the one you ship News
A top-ranked paper shows that in modern RL training, improving the model you optimize does not guarantee improving the model you actually deploy - because the two run on different engines that disagree on probabilities.
Group Relative Policy Optimization (GRPO) Lesson
GRPO is the reinforcement learning method behind most modern reasoning models: instead of training a separate 'critic' network to judge answers, it samples a whole group of answers to each question and scores each one by how it compares to the group's average.
ICML's top paper says diffusion language models sabotage their own best feature News
ICML 2026 gave an Outstanding Paper award to work showing that diffusion language models waste their much-touted any-order generation by skipping the hardest, most decision-critical words -- and that forcing a plain left-to-right order during training fixes it.
Two new papers push 'on-policy distillation' to fix privileged teachers and merge specialist skills News
DOPD and MOPD advance on-policy distillation -- training a student on its own outputs -- with DOPD routing supervision to avoid a 'privilege illusion' and MOPD merging multiple specialist RL teachers into one model without cross-domain interference.
On-Policy vs Off-Policy Learning Lesson
On-policy learning trains a model on data generated by its own current behavior, while off-policy learning trains it on data generated by something else -- an old version, a different policy, or a fixed dataset -- and the choice shapes how stable, sample-efficient, and reliable the training is.
'Dockerless' verifies AI code patches by reading the repo instead of running it News
A new method called Dockerless judges whether an AI's code patch is correct by having an agent explore the repository for evidence rather than executing tests in a Docker container, enabling a fully environment-free training pipeline for coding agents.
Chain-of-thought: why making an AI think out loud makes it smarter Lesson
Asking a model to work through a problem step by step, instead of blurting an answer, dramatically improves it on hard tasks. Here is why that simple trick works, what it really buys the model, and where it backfires.
Teaching AI with rewards — minus the expensive second model that grades it News
The standard way to polish a model with rewards quietly runs a second 'critic' model alongside it. A new method derives the critic's judgment from the model itself, dropping the extra cost.
Polishing AI by looking inside its 'mind' instead of just thumbs-up, thumbs-down News
Reward training usually treats the model as a black box — thumbs up, thumbs down, hope for the best. A new method peers inside to see why an answer was preferred, and shapes the lesson on purpose.
The little words that keep AI from getting boring News
Rewarding a reasoning model too hard makes it repetitive — and the casualties are tiny words like "but" and "instead" that let it branch to a better thought. A near-free fix protects them.
Reward-based fine-tuning (RLHF and RLVR) Lesson
After a model is first trained, it gets "polished" by rewarding good answers. Here's what that phase is, why it works, and the failure mode where models get repetitive and dull.
Faster AI training by quietly cloning the model News
Teaching a model with rewards is slow because it has to write out endless practice answers. A new trick: make a cheap, shrunk-down copy of the model to crank those out faster.
Crediting an AI for the right steps — without a second model to judge them News
When you reward an AI for a good final answer, it's hard to know which of its steps earned the credit. The usual fix is training a second 'judge' model. This skips that.