# What is GELU? Definition & Technical Mechanics

> **Source:** [SPIDITS AI AI Glossary](https://spidits.com/ai-glossary)  
> **Category:** Mathematical Foundations  
> **Canonical Citation:** [https://spidits.com/ai-glossary/gelu](https://spidits.com/ai-glossary/gelu)

## Definition
GELU (Gaussian Error Linear Unit) is a smooth activation function that scales input values by the cumulative distribution function of the standard normal distribution, commonly used in BERT and modern Transformers.

## Architectural Mechanics & Deep Dive
GeLU (Gaussian Error Linear Unit) is a modern activation function widely used in Transformer architectures like BERT and GPT. It weights inputs by their probability under a cumulative Gaussian distribution. Unlike ReLU which cuts off negative values abruptly, GeLU provides a smooth, non-linear curve that allows gradients to flow for small negative inputs, improving model convergence.

## Industry Applications & Real-World Use Cases
Transformer architecture activation, neural network design, and model training.

## Frequently Asked Questions

### What is the benefit of GELU over standard ReLU?
GELU is differentiable at all points and allows a small non-zero gradient for negative inputs, preventing "dead neuron" issues.

### Which model popularized the use of GELU?
BERT (Bidirectional Encoder Representations from Transformers) in 2018.

## Related Terminology
- [activation-function](https://spidits.com/ai-glossary/activation-function)
- [bert](https://spidits.com/ai-glossary/bert)
- [transformer](https://spidits.com/ai-glossary/transformer)

---
*From the SPIDITS AI Knowledge Base (336+ terms indexed). Explore full technical definitions: [https://spidits.com/ai-glossary](https://spidits.com/ai-glossary)*