Dense Supervision, Sparse Updates: On the Sparsity and Geometry of On-Policy Distillation

On-policy distillation (\textsc{OPD}) has recently become a prominent post-training recipe as it combines two desirable ingredients: on-policy student trajectories and dense teacher supervision, yet how this hybrid changes a model's parameters remains unclear. Across several language and vision-lang...