On-Policy Self-Distillation without Any Supervision
Yijiang Li, Bingyang Wang, Yijun Liang, et al.
Proposes U-OPSD, an unsupervised on-policy self-distillation method that uses only a model's own generations and majority voting to improve reasoning without external supervision.