# What is Multi-Query Attention? Definition & Technical Mechanics

> **Source:** [SPIDITS AI AI Glossary](https://spidits.com/ai-glossary)  
> **Category:** Neural Architectures  
> **Canonical Citation:** [https://spidits.com/ai-glossary/mqa](https://spidits.com/ai-glossary/mqa)

## Definition
Multi-Query Attention (MQA) is an attention architecture where all query heads share a single Key and Value head to minimize KV cache storage.

## Architectural Mechanics & Deep Dive
Multi-Query Attention (MQA) is an attention mechanism optimization where all query heads share a single Key-Value head. MQA drastically reduces the size of the Key-Value (KV) cache stored in GPU memory during generation, accelerating inference throughput and allowing for larger batch sizes at the expense of a minor degradation in model capacity.

## Industry Applications & Real-World Use Cases
Extreme scale caching, low-end edge device inference.

## Frequently Asked Questions

### What is the main benefit of MQA?
It drastically shrinks the memory capacity needed for key-value storage.

### Does MQA reduce model quality?
Yes, sharing a single K/V head across all query heads causes slight quality degradation.

## Related Terminology
- [attention-mechanism](https://spidits.com/ai-glossary/attention-mechanism)
- [kv-cache](https://spidits.com/ai-glossary/kv-cache)
- [gqa](https://spidits.com/ai-glossary/gqa)

---
*From the SPIDITS AI Knowledge Base (336+ terms indexed). Explore full technical definitions: [https://spidits.com/ai-glossary](https://spidits.com/ai-glossary)*