Firehose

Filtered to tagged “benchmarks” · clear filters

All PeopleCompaniesPapersPodcastsHacker News

Browse by tag

13 NOV 2026 · Paper

This paper recovers vehicle trajectories from CAN signals in naturalistic driving and presents a new dataset for evaluating such methods. Practitioners might care because it shows how to accurately track vehicle movements even when GPS data is removed, which could be useful for usage-based insurance and location-based services.

13 NOV 2026 · Paper

This paper creates a benchmark for large language models to evaluate their performance in finance, specifically in Japan, and shows that some models can excel in this domain even with limited external knowledge.

13 NOV 2026 · Paper

This paper introduces a new benchmark for web agents that can perform tasks across multiple web applications and modalities, including text, vision, audio, and video. Practitioners can care about this benchmark because it provides a comprehensive evaluation framework for web agents that can handle complex, real-world tasks.

12 NOV 2026 · Paper

This paper explores how video models can be used to render 3D worlds created by code, and whether they can accurately preserve the world's structure and appearance over time. Practitioners might care because reliable generative rendering is crucial for applications like video games and animation.

12 NOV 2026 · Paper

This paper explores the capabilities of decision models in long-horizon agents, examining whether a poorly performing model can still be effective in teaching a policy through reinforcement learning. Practitioners might care about the implications for designing more robust and effective reinforcement learning systems.

11 NOV 2026 · Paper

This paper introduces SO-Arena, a benchmark that evaluates the capabilities and reliability of System One models, which are fast decision-making models, and shows that even the best-performing models can make unreliable decisions, highlighting the need for a more comprehensive understanding of these models' strengths and limitations.