# What is Self-Attention? Definition & Technical Mechanics

> **Source:** [SPIDITS AI AI Glossary](https://spidits.com/ai-glossary)  
> **Category:** Neural Architectures  
> **Canonical Citation:** [https://spidits.com/ai-glossary/self-attention](https://spidits.com/ai-glossary/self-attention)

## Definition
Self-Attention (or scaled dot-product attention) is an attention mechanism that relates different positions of a single sequence to compute a representation of the same sequence, allowing the model to calculate context dynamically.

## Architectural Mechanics & Deep Dive
Self-attention is the mathematical mechanism in Transformers that computes relative importance scores between all tokens in a sequence simultaneously. By calculating Query, Key, and Value vector projections for each word and scoring their alignment, self-attention enables the model to capture complex, long-range dependencies and multi-layered relationships without relying on recurrent loops.

## Industry Applications & Real-World Use Cases
Transformer encoder-decoder blocks, context calculations, and sequence modeling.

## Frequently Asked Questions

### How does self-attention calculate relationships between words?
By projecting each word token into three vectors: Query (Q), Key (K), and Value (V). It multiplies Q and K to get a similarity score, scales the result, applies softmax to get attention weights, and multiplies by V.

### What is the scaling factor in scaled dot-product attention?
It divides the dot products of Query and Key vectors by the square root of their dimension size. This prevents the dot products from growing extremely large in high dimensions, which could push the softmax function into regions with vanishing gradients.

## Related Terminology
- [attention-mechanism](https://spidits.com/ai-glossary/attention-mechanism)
- [transformer](https://spidits.com/ai-glossary/transformer)
- [multi-head-attention](https://spidits.com/ai-glossary/multi-head-attention)

---
*From the SPIDITS AI Knowledge Base (336+ terms indexed). Explore full technical definitions: [https://spidits.com/ai-glossary](https://spidits.com/ai-glossary)*