This paper introduces SO-Arena, a benchmark that evaluates the capabilities and reliability of System One models, which are fast decision-making models, and shows that even the best-performing models can make unreliable decisions, highlighting the need for a more comprehensive understanding of these models' strengths and limitations.
Firehose
Filtered to Papers, tagged “evaluation methods” · clear filters
Browse: People · Companies · Papers · Podcasts · Hacker News · Deep dives