# What is Stochastic Gradient Descent? Definition & Technical Mechanics

> **Source:** [SPIDITS AI AI Glossary](https://spidits.com/ai-glossary)  
> **Category:** Mathematical Foundations  
> **Canonical Citation:** [https://spidits.com/ai-glossary/stochastic-gradient-descent](https://spidits.com/ai-glossary/stochastic-gradient-descent)

## Definition
Stochastic Gradient Descent (SGD) is an optimization algorithm that updates a model's weights using the gradient calculated from a single randomly chosen training sample (or a small batch) rather than the entire dataset.

## Architectural Mechanics & Deep Dive
Stochastic Gradient Descent (SGD) is a foundational optimization algorithm. Instead of calculating gradients across the entire dataset (which is computationally prohibitive), SGD estimates the gradient using a single random sample or mini-batch per iteration, updating weights rapidly to speed up convergence.

## Industry Applications & Real-World Use Cases
Training deep learning models, online learning systems, and scaling optimization on massive datasets.

## Frequently Asked Questions

### How does SGD differ from Batch Gradient Descent?
Batch Gradient Descent computes gradients on the entire dataset before making one update, which is slow and memory-intensive. SGD updates weights much faster by using single samples or mini-batches.

### Why does SGD have a noisy or fluctuating optimization path?
Because it estimates the true gradient using only a subset of data, which introduces variance. This noise can actually help the optimizer escape poor local minima.

## Related Terminology
- [gradient-descent](https://spidits.com/ai-glossary/gradient-descent)
- [adam-optimizer](https://spidits.com/ai-glossary/adam-optimizer)
- [learning-rate](https://spidits.com/ai-glossary/learning-rate)

---
*From the SPIDITS AI Knowledge Base (336+ terms indexed). Explore full technical definitions: [https://spidits.com/ai-glossary](https://spidits.com/ai-glossary)*