Firehose

Filtered to Papers, tagged “self-distillation” · clear filters

Browse: People · Companies · Papers · Podcasts · Hacker News · Deep dives

All People Companies Papers Podcasts Hacker News

Browse by tag

continual learning 31 reinforcement learning 10 language models 4 robotics 4 agents 3 large language models 3 recursive self-improvement 3 vision-language models 3

17 SEP 2026 · Paper

What Does Privileged Information Add to On-Policy Self-Distillation?

This paper investigates whether giving a language model extra information, such as a worked solution, improves its learning through on-policy self-distillation. A practitioner might care about how to optimize this technique for better performance.

self-distillation on-policy learning language models reasoning capabilities transfer learning