Firehose

Filtered to Papers, tagged “SOAP” · clear filters

All PeopleCompaniesPapersPodcastsHacker News

Browse by tag

12 NOV 2026 · Paper

This paper proposes a more efficient version of the SOAP optimizer, called Factored One-sided Adam-Moment (FOAM), which retains the benefits of SOAP while using less memory and running faster. Practitioners caring about optimizing large language models may benefit from this more efficient optimizer.