post-training
Direct Preference Optimization: skipping the reward model entirely Lesson
Direct Preference Optimization trains a language model on human preference pairs without ever building a separate reward model or running reinforcement learning, by showing mathematically that the model can serve as its own reward function.
Z.ai changed only the post-training, and the model learned to find exploits News
Z.ai released GLM-5.3 on August 14 using the same base model as GLM-5.2, with every gain coming from post-training, and the largest jump was in finding and exploiting software vulnerabilities.
A model improved itself by training only where it disagreed with itself News
U-OPSD drops the teacher entirely: it samples several attempts, takes a majority vote as a stand-in answer, then trains only on the attempts that disagreed with that vote, matching or beating methods that use real ground-truth labels.
Grafting a Verified Solution Cache Lets a Frozen Model Skip Fine-Tuning Entirely News
A new paper shows a frozen small language model's accuracy on a hard math test can jump from 80.0% to 93.3% simply by grafting a byte-exact cache of verified solutions into it, with no weight changes at all.
Qwen used human-feedback training to make its image AI follow directions better News
A new recipe applies the same reinforcement-learning approach that polished chatbots to an image generator, then merges separate skill models into one - improving how faithfully it follows prompts and edits.
SLAI T-Rex Tool
The public workflow behind a full-parameter Ascend post-training run on a DeepSeek-V4-family model: FP8-to-BF16-to-Megatron checkpoint conversion, launch templates, and inspectable data-construction pipelines for continued pre-training and supervised fine-tuning. The production engine and custom kernels are withheld.