{"id":"arxiv:2503.11816","kind":"preprint","title":"Key, Value, Compress: A Systematic Exploration of KV Cache Compression Techniques","authors":["Neusha Javidnia","Bita Darvish Rouhani","Farinaz Koushanfar"],"author_count":3,"year":2025,"published":"2025-03-14","venue":null,"cited_by":null,"doi":null,"arxiv_id":"2503.11816","nct_id":null,"source_url":"https://arxiv.org/abs/2503.11816","archive_url":"https://space-kitty.com/papers/arxiv:2503.11816","updated":"2025-04-22","pmid":null,"pmcid":null,"openalex_id":null,"pdf_url":"https://arxiv.org/pdf/2503.11816","abstract":"Large language models (LLMs) have demonstrated exceptional capabilities in generating text, images, and video content. However, as context length grows, the computational cost of attention increases quadratically with the number of tokens, presenting significant efficiency challenges. This paper presents an analysis of various Key-Value (KV) cache compression strategies, offering a comprehensive taxonomy that categorizes these methods by their underlying principles and implementation techniques. Furthermore, we evaluate their impact on performance and inference latency, providing critical insights into their effectiveness. Our findings highlight the trade-offs involved in KV cache compression and its influence on handling long-context scenarios, paving the way for more efficient LLM implementations.","abstract_license":"CC0-1.0 (arXiv descriptive metadata)","license":null,"is_open_access":false,"retracted":false,"peer_reviewed":false,"categories":["cs.CL"],"details":null,"provenance":"arxiv","attribution":"Abstract: arXiv descriptive metadata (CC0). Thank you to arXiv for use of its open access interoperability. This service was not reviewed or approved by, nor does it necessarily express or reflect the policies or opinions of, arXiv.","topics":[],"cited_in_dossiers":[]}