on-policy
A new distillation method lets the teacher model interrupt the student mid-thought News
Researchers found that when a student model starts reasoning down a wrong path, its teacher's next word tends to be a redirection like But or Wait, and turned that disagreement into an automatic trigger for the teacher to briefly take over.
Two new papers push 'on-policy distillation' to fix privileged teachers and merge specialist skills News
DOPD and MOPD advance on-policy distillation -- training a student on its own outputs -- with DOPD routing supervision to avoid a 'privilege illusion' and MOPD merging multiple specialist RL teachers into one model without cross-domain interference.
On-Policy vs Off-Policy Learning Lesson
On-policy learning trains a model on data generated by its own current behavior, while off-policy learning trains it on data generated by something else -- an old version, a different policy, or a fixed dataset -- and the choice shapes how stable, sample-efficient, and reliable the training is.