Firehose

Filtered to Hacker News, tagged “large language models” · clear filters

All PeopleCompaniesPapersPodcastsHacker News

Browse by tag

9 NOV 2026 · Hacker News · 79 pts · 12 comments ↗

The KV cache is a shared memory-mapped pool that stores key-value pairs for the Llama model, with a capacity of 128 MiB by default. It is used to cache query and key values during the inference process, allowing for efficient access and reuse of these values across multiple requests. AI summary