# Towards Efficient Large Language Model Serving: A Survey on System-Aware KV Cache Optimization

Preprint · Jiantong Jiang, Peiyu Yang, Rui Zhang, Feng Liu

- Archive id: arxiv:2607.08057
- Published: 2026-07-09
- Venue: Findings of the Association for Computational Linguistics: ACL 2026 (pp. 38450-38476)
- DOI: https://doi.org/10.18653/v1/2026.findings-acl.1916
- arXiv: https://arxiv.org/abs/2607.08057
- Source URL: https://arxiv.org/abs/2607.08057
- PDF: https://arxiv.org/pdf/2607.08057
- Provenance: arxiv

## Abstract

Despite the rapid advancements of large language models (LLMs), LLM serving systems remain memory-intensive and costly. The key-value (KV) cache, which stores KV tensors during autoregressive decoding, is crucial for enabling low-latency, high-throughput LLM inference serving. In this survey, we focus on system-aware KV infrastructure for serving LLMs (abbreviated as sKis). We revisit recent work from a system behavior perspective, organizing existing efforts into three dimensions: execution and scheduling (temporal), placement and migration (spatial), and representation and retention (structural). Furthermore, we analyze cross-behavior co-design affinity and behavior-objective links, highlighting future opportunities. Our work systematizes a rapidly evolving area, providing a foundation for understanding and innovating KV cache designs in modern LLM serving infrastructure.

_Abstract reproduced under CC0-1.0 (arXiv descriptive metadata)._

---
Space Kitty Archive · https://space-kitty.com · API: https://space-kitty.com/api/v1 · MCP: https://space-kitty.com/mcp · Premium: $0.20/request via x402 (Base Sepolia (testnet)). Metadata: arXiv (CC0), OpenAlex (CC0), Europe PMC (CC BY/CC0 works), ClinicalTrials.gov.