# What is KV Cache Eviction? Definition & Technical Mechanics

> **Source:** [SPIDITS AI AI Glossary](https://spidits.com/ai-glossary)  
> **Category:** Hardware & Infrastructure  
> **Canonical Citation:** [https://spidits.com/ai-glossary/kv-cache-eviction](https://spidits.com/ai-glossary/kv-cache-eviction)

## Definition
KV Cache Eviction is a memory management technique that removes less important key-value states from GPU memory during long text generation. This prevents out-of-memory errors and keeps sequence processing fast.

## Architectural Mechanics & Deep Dive
KV Cache Eviction is a memory management technique that dynamically drops less important keys and values from the GPU memory cache during long-context generation. By evaluating token attention weights or frequency metrics, the eviction policy retains critical context (like attention sinks and recent query history) while discarding redundant states, preventing out-of-memory errors.

## Industry Applications & Real-World Use Cases
Long context generation, persistent multi-user serving, and hardware optimization

## Frequently Asked Questions

### How does cache eviction choose what to remove?
Using metrics like attention scores, age (least recently used), or semantic importance to discard non-essential tokens while preserving attention sinks.

### Why is KV cache memory a bottleneck?
Because the cache size scales linearly with both batch size and context length, quickly consuming available GPU VRAM.

## Related Terminology
- [kv-cache](https://spidits.com/ai-glossary/kv-cache)
- [attention-sink](https://spidits.com/ai-glossary/attention-sink)
- [gpu](https://spidits.com/ai-glossary/gpu)

---
*From the SPIDITS AI Knowledge Base (300+ terms indexed). Explore full technical definitions: [https://spidits.com/ai-glossary](https://spidits.com/ai-glossary)*