# What is SGD with Momentum? Definition & Technical Mechanics

> **Source:** [SPIDITS AI AI Glossary](https://spidits.com/ai-glossary)  
> **Category:** Mathematical Foundations  
> **Canonical Citation:** [https://spidits.com/ai-glossary/sgd-with-momentum](https://spidits.com/ai-glossary/sgd-with-momentum)

## Definition
SGD with Momentum is an extension of Stochastic Gradient Descent that accelerates weight updates in the relevant direction by adding a fraction of the previous update vector to the current step.

## Architectural Mechanics & Deep Dive
Stochastic Gradient Descent (SGD) with Momentum is an optimization algorithm that accelerates gradient updates by accumulating a fraction of past gradient vectors. This mathematical momentum acts like a ball rolling down a hill, smoothing out noisy gradient paths, accelerating convergence in steep ravines, and helping the optimizer escape shallow local minima.

## Industry Applications & Real-World Use Cases
Accelerating neural network convergence, smoothing noisy gradient updates, and training computer vision models.

## Frequently Asked Questions

### What analogy is commonly used to describe momentum in SGD?
A heavy ball rolling down a hill, which gathers speed and momentum as it descends, passing over minor bumps or local flat regions.

### What is the role of the momentum coefficient parameter?
It determines how much of the past gradient history is kept. Usually set close to 0.9, it acts as a friction parameter that dampens oscillations.

## Related Terminology
- [stochastic-gradient-descent](https://spidits.com/ai-glossary/stochastic-gradient-descent)
- [gradient-descent](https://spidits.com/ai-glossary/gradient-descent)
- [adam-optimizer](https://spidits.com/ai-glossary/adam-optimizer)

---
*From the SPIDITS AI Knowledge Base (336+ terms indexed). Explore full technical definitions: [https://spidits.com/ai-glossary](https://spidits.com/ai-glossary)*