# What is Transformer? Definition & Technical Mechanics

> **Source:** [SPIDITS AI AI Glossary](https://spidits.com/ai-glossary)  
> **Category:** Neural Architectures  
> **Canonical Citation:** [https://spidits.com/ai-glossary/transformer](https://spidits.com/ai-glossary/transformer)

## Definition
A Transformer is a deep learning neural network architecture introduced in 2017 by Google researchers, based entirely on self-attention mechanisms. It processes sequential inputs in parallel, capturing long-range dependencies and serving as the foundational engine for all modern LLMs.

## Architectural Mechanics & Deep Dive
The Transformer is a neural network architecture introduced in the paper "Attention Is All You Need." Based entirely on self-attention mechanisms and feedforward layers, the Transformer processes input sequences in parallel, capturing complex long-range dependencies, forming the foundation of modern generative AI.

## Industry Applications & Real-World Use Cases
Large Language Models (GPT, Gemini, Claude), neural machine translation, and text-to-image foundation models.

## Frequently Asked Questions

### Why did Transformers replace recurrent architectures like LSTMs?
Recurrent models process tokens sequentially, which makes them slow and difficult to parallelize on GPUs. Transformers process entire sequences simultaneously, allowing them to train on massive web-scale datasets.

### What is the role of positional encoding in Transformers?
Since Transformers process all sequence tokens simultaneously, they have no built-in sense of order. Positional encoding adds coordinate markers to the token vectors to convey the order of words.

## Related Terminology
- [attention-mechanism](https://spidits.com/ai-glossary/attention-mechanism)
- [self-attention](https://spidits.com/ai-glossary/self-attention)
- [llm](https://spidits.com/ai-glossary/llm)
- [bert](https://spidits.com/ai-glossary/bert)

---
*From the SPIDITS AI Knowledge Base (336+ terms indexed). Explore full technical definitions: [https://spidits.com/ai-glossary](https://spidits.com/ai-glossary)*