# What is Grouped-Query Attention? Definition & Technical Mechanics

> **Source:** [SPIDITS AI AI Glossary](https://spidits.com/ai-glossary)  
> **Category:** Neural Architectures  
> **Canonical Citation:** [https://spidits.com/ai-glossary/gqa](https://spidits.com/ai-glossary/gqa)

## Definition
Grouped-Query Attention (GQA) is an attention query layout grouping query heads to share a single Key and Value head, reducing the memory footprint of the KV cache.

## Architectural Mechanics & Deep Dive
Grouped-Query Attention (GQA) is an optimization technique for Transformer models that speeds up inference by grouping query heads to share a single key-value head. GQA acts as a middle ground between Multi-Head Attention (MHA) and Multi-Query Attention (MQA), delivering near-MHA quality while significantly reducing memory bandwidth usage and accelerating KV cache operations.

## Industry Applications & Real-World Use Cases
Lightweight model inference, mobile device deployment, and LLaMA 3 architectures.

## Frequently Asked Questions

### Why use GQA over Multi-Head Attention?
GQA dramatically shrinks the KV cache size, improving throughput with minor quality tradeoffs.

### How does it compare to Multi-Query Attention (MQA)?
GQA offers a balance, providing higher model quality than MQA and better generation speed than MHA.

## Related Terminology
- [attention-mechanism](https://spidits.com/ai-glossary/attention-mechanism)
- [kv-cache](https://spidits.com/ai-glossary/kv-cache)
- [llama](https://spidits.com/ai-glossary/llama)

---
*From the SPIDITS AI Knowledge Base (336+ terms indexed). Explore full technical definitions: [https://spidits.com/ai-glossary](https://spidits.com/ai-glossary)*