# What is Multi-Head Attention? Definition & Technical Mechanics

> **Source:** [SPIDITS AI AI Glossary](https://spidits.com/ai-glossary)  
> **Category:** Neural Architectures  
> **Canonical Citation:** [https://spidits.com/ai-glossary/multi-head-attention](https://spidits.com/ai-glossary/multi-head-attention)

## Definition
Multi-Head Attention is an attention layout in Transformers that splits query, key, and value vectors into multiple subspaces, allowing the model to attend to information from different representation coordinates simultaneously.

## Architectural Mechanics & Deep Dive
Multi-head attention is the core component of the Transformer architecture. It splits the input vectors into multiple subspaces, allowing the model to perform attention calculations in parallel across multiple heads. This enables the network to attend to information from different representation subspaces and positions simultaneously, capturing complex context.

## Industry Applications & Real-World Use Cases
Transformer block operations, sequence correlation mapping, and LLM design.

## Frequently Asked Questions

### Why use multiple attention heads instead of one?
A single attention head averages out focus. Multi-head attention allows the model to simultaneously look at different tokens (e.g. grammar structure and semantic pronouns).

### What is the output of the multi-head attention layer?
The concatenated outputs of each individual attention head, projected back to the original embedding size.

## Related Terminology
- [attention-mechanism](https://spidits.com/ai-glossary/attention-mechanism)
- [gqa](https://spidits.com/ai-glossary/gqa)
- [transformer](https://spidits.com/ai-glossary/transformer)

---
*From the SPIDITS AI Knowledge Base (336+ terms indexed). Explore full technical definitions: [https://spidits.com/ai-glossary](https://spidits.com/ai-glossary)*