post-training
The day's most-upvoted paper turns a model's own routing logs into its next training set News
A team calling itself NeoHorse released NeoHorse-1, a system that records which model handled each user request and how it went, then converts those logs into training data for the next round, closing an evaluation-to-training loop the authors present as a prototype path toward recursive self-improvement.
Three papers landed the same day arguing you should build the world, not the model News
EnvHarness, FACET, and SPADE all took the top spots on Hugging Face's daily paper list with the same underlying move, shifting effort from making the agent smarter to manufacturing the environments the agent practices in, with FACET releasing 6,020 ready-made terminal tasks.
Reinforcement learning changed only 1 to 3 percent of a model's tokens News
A token-level analysis found that reinforcement learning's entire benefit to reasoning models is concentrated at a tiny number of uncertain decision points, and a training-free method reproduced most of it for roughly a thousandth of the cost.
Direct Preference Optimization: skipping the reward model entirely Lesson
Direct Preference Optimization trains a language model on human preference pairs without ever building a separate reward model or running reinforcement learning, by showing mathematically that the model can serve as its own reward function.
Z.ai changed only the post-training, and the model learned to find exploits News
Z.ai released GLM-5.3 on August 14 using the same base model as GLM-5.2, with every gain coming from post-training, and the largest jump was in finding and exploiting software vulnerabilities.
A model improved itself by training only where it disagreed with itself News
U-OPSD drops the teacher entirely: it samples several attempts, takes a majority vote as a stand-in answer, then trains only on the attempts that disagreed with that vote, matching or beating methods that use real ground-truth labels.
Grafting a Verified Solution Cache Lets a Frozen Model Skip Fine-Tuning Entirely News
A new paper shows a frozen small language model's accuracy on a hard math test can jump from 80.0% to 93.3% simply by grafting a byte-exact cache of verified solutions into it, with no weight changes at all.
Qwen used human-feedback training to make its image AI follow directions better News
A new recipe applies the same reinforcement-learning approach that polished chatbots to an image generator, then merges separate skill models into one - improving how faithfully it follows prompts and edits.
SLAI T-Rex Tool
The public workflow behind a full-parameter Ascend post-training run on a DeepSeek-V4-family model: FP8-to-BF16-to-Megatron checkpoint conversion, launch templates, and inspectable data-construction pipelines for continued pre-training and supervised fine-tuning. The production engine and custom kernels are withheld.
NeoHorse-1 Tool
Code and models for a routing harness that logs which model handled each request and how it went, then converts those logs into the next round of training data. Released alongside the paper as a working implementation of the evaluation-to-training loop.