# What is Dense Model? Definition & Technical Mechanics

> **Source:** [SPIDITS AI AI Glossary](https://spidits.com/ai-glossary)  
> **Category:** Neural Architectures  
> **Canonical Citation:** [https://spidits.com/ai-glossary/dense-model](https://spidits.com/ai-glossary/dense-model)

## Definition
A Dense Model is a neural network architecture where 100% of the model's parameters are activated and calculated for every single token processed, representing the traditional design of deep neural networks.

## Architectural Mechanics & Deep Dive
A dense model (or dense neural network) is an architecture where every parameter (weight) is actively utilized for every forward and backward pass. This stands in contrast to sparse models, which activate only a subset of parameters for a given input. While dense models are computationally efficient on modern GPU hardware due to highly optimized matrix multiplication operations, they hit scaling limits as models grow to hundreds of billions of parameters.

## Industry Applications & Real-World Use Cases
Standard LLMs (like early GPT-3), standard CNN vision models, and baseline neural networks.

## Frequently Asked Questions

### How does a dense model differ from a sparse model?
Dense models activate all parameters for every token. Sparse models (like MoE) route tokens to specific subsets of parameters, reducing computation per token.

### What is the main drawback of scaling dense models?
Higher parameter counts lead to quadratic compute cost increases during both training and inference.

## Related Terminology
- [sparse-model](https://spidits.com/ai-glossary/sparse-model)
- [mixture-of-experts](https://spidits.com/ai-glossary/mixture-of-experts)
- [neural-network](https://spidits.com/ai-glossary/neural-network)

---
*From the SPIDITS AI Knowledge Base (336+ terms indexed). Explore full technical definitions: [https://spidits.com/ai-glossary](https://spidits.com/ai-glossary)*