# What is Mixture of Experts? Definition & Technical Mechanics

> **Source:** [SPIDITS AI AI Glossary](https://spidits.com/ai-glossary)  
> **Category:** Neural Architectures  
> **Canonical Citation:** [https://spidits.com/ai-glossary/mixture-of-experts](https://spidits.com/ai-glossary/mixture-of-experts)

## Definition
Mixture of Experts (MoE) is a neural network design that scales model parameters without increasing compute cost. Instead of activating the entire network for every token, MoE routes inputs to specialized sub-networks ("experts") using a gating router.

## Architectural Mechanics & Deep Dive
Mixture of Experts (MoE) is a model architecture that scales parameters while maintaining constant compute costs. It splits layers into multiple independent "expert" neural networks. A gating network routes each token to the most relevant experts (e.g., 2 out of 8 experts per token), activating only a fraction of total model weights per forward pass.

## Industry Applications & Real-World Use Cases
High-parameter language models (like Mixtral, GPT-4) and resource-efficient inference

## Frequently Asked Questions

### What is active parameter count vs. total parameter count in MoE?
Total parameter count is the size of all experts combined. Active parameter count is the size of only the experts triggered for a specific token (e.g. 2 out of 8 experts).

### What is a Mixture of Experts (MoE) model and how does it lower inference costs?
An MoE model routes inputs dynamically to specialized expert sub-networks using a gating router. Because only a subset of parameters are active per token (e.g., 2 out of 8 experts), it provides massive parameter capacity with the fast inference times of smaller models.

## Related Terminology
- [llm](https://spidits.com/ai-glossary/llm)
- [transformer](https://spidits.com/ai-glossary/transformer)
- [inference](https://spidits.com/ai-glossary/inference)

---
*From the SPIDITS AI Knowledge Base (336+ terms indexed). Explore full technical definitions: [https://spidits.com/ai-glossary](https://spidits.com/ai-glossary)*