Firehose

This paper investigates whether giving a language model extra information, such as a worked solution, improves its learning through on-policy self-distillation. A practitioner might care about how to optimize this technique for better performance.

This paper proposes a new method for estimating confidence in language models, called XConf, which uses the model's past experiences to inform its confidence, rather than just relying on the current inference process. Practitioners might care about this because it could lead to more reliable and trustworthy deployment of language models.

This paper tests the robustness of rubrics generated by language models as reward signals in reinforcement learning, finding that even generic rubrics can be exploited 64% of the time, while tailored rubrics can be used to create fake answers. Practitioners should care because this can lead to biased grading and evaluation.

This paper introduces a fast and efficient post-hoc defense against a type of attack that can bypass safety features in language models, allowing the model to continue functioning but with compromised security. Practitioners caring about model security may be interested in this approach as it can provide an additional layer of protection without requiring significant computational resources.