Unified KV Cache Compression Methods for Auto-Regressive Models

LLM KV cache compression made easy

Awesome-LLM-KV-Cache: A curated list of 📙Awesome LLM KV Cache Papers with Codes.

[NeurIPS'25 Oral] Query-agnostic KV cache eviction: 3–4× reduction in memory and 2× decrease in latency (Qwen3/2.5, Gemma3, LLaMA3)

Block Transformer: Global-to-Local Language Modeling for Fast Inference (NeurIPS 2024)

[ICLR 2025] Palu: Compressing KV-Cache with Low-Rank Projection

Pytorch implementation for "Compressed Context Memory For Online Language Model Interaction" (ICLR'24)

This is the official repo of "QuickLLaMA: Query-aware Inference Acceleration for Large Language Models"

xKV: Cross-Layer SVD for KV-Cache Compression

(ACL 2025 oral) SCOPE: Optimizing KV Cache Compression in Long-context Generation

Accurate and fast KV cache compression with a gating mechanism

LAVa: Layer-wise KV Cache Eviction with Dynamic Budget Allocation

Repository for the paper: https://arxiv.org/abs/2510.00231