# From Tensor Buffer to Distributed Memory Hierarchy: A Survey of KV Cache Management for LLM Serving

Preprint · Jie Li, Tongyang Wang, Yong Chen

- Archive id: arxiv:2607.02574
- Published: 2026-06-30
- arXiv: https://arxiv.org/abs/2607.02574
- Source URL: https://arxiv.org/abs/2607.02574
- PDF: https://arxiv.org/pdf/2607.02574
- Provenance: arxiv

## Abstract

The key-value (KV) cache has become a first-order memory object in LLM serving rather than a temporary per-request tensor. This survey classifies more than thirty KV-management systems and frameworks using four axes: locality, lifetime, ownership, and substrate. The axes reveal five architectural archetypes -- local-paged, disaggregated-pipeline, shared-store, memory-pool, and hybrid-tier. Once workload and hardware are fixed, ownership accounts for much of the remaining design variance among distributed systems. The survey also audits current evaluations and identifies seven missing KV-specific measurements, linking them to open problems in fault tolerance, isolation, tiered eviction, speculative decoding, MoE serving, and shared-cache semantics.

_Abstract reproduced under CC0-1.0 (arXiv descriptive metadata)._

---
Space Kitty Archive · https://space-kitty.com · API: https://space-kitty.com/api/v1 · MCP: https://space-kitty.com/mcp · Premium: $0.20/request via x402 (Base Sepolia (testnet)). Metadata: arXiv (CC0), OpenAlex (CC0), Europe PMC (CC BY/CC0 works), ClinicalTrials.gov.