Firehose

Filtered to tagged “inference” · clear filters

All PeopleCompaniesPapersPodcastsHacker News

Browse by tag

13 NOV 2026 · Deep Learning Weekly

Claude Fable 5.2, Inference Performance Index: changing the way we measure inference capacity, a paper on Tools Need Not Speak in Text: Latent Context Compression for Agents, and many more!

9 NOV 2026 · Hacker News · 79 pts · 12 comments ↗

The KV cache is a shared memory-mapped pool that stores key-value pairs for the Llama model, with a capacity of 128 MiB by default. It is used to cache query and key values during the inference process, allowing for efficient access and reuse of these values across multiple requests. AI summary