# What is Gradient Descent? Definition & Technical Mechanics

> **Source:** [SPIDITS AI AI Glossary](https://spidits.com/ai-glossary)  
> **Category:** Mathematical Foundations  
> **Canonical Citation:** [https://spidits.com/ai-glossary/gradient-descent](https://spidits.com/ai-glossary/gradient-descent)

## Definition
Gradient Descent is an optimization algorithm used to minimize a model's loss function during training. It iteratively calculates the slope (gradient) of the error surface and updates model parameters (weights) in the direction of the steepest descent.

## Architectural Mechanics & Deep Dive
Gradient descent is an iterative optimization algorithm used to minimize a model's cost function. By calculating the gradient (derivative) of the loss function with respect to model parameters, it updates parameters in the opposite direction of the gradient. This step-by-step adjustment, scaled by the learning rate, guides the model toward the global or local minimum of the loss surface.

## Industry Applications & Real-World Use Cases
Neural network training, parameter optimization, and regression models

## Frequently Asked Questions

### What is the learning rate in gradient descent?
A hyperparameter determining the step size taken towards the minimum. Too large can cause divergence; too small makes training extremely slow.

### What is Stochastic Gradient Descent (SGD)?
A variant of gradient descent that calculates gradients and updates weights using only a single random training sample (or batch) at a time, speeding up updates.

## Related Terminology
- [backpropagation](https://spidits.com/ai-glossary/backpropagation)
- [loss-function](https://spidits.com/ai-glossary/loss-function)
- [hyperparameter](https://spidits.com/ai-glossary/hyperparameter)

---
*From the SPIDITS AI Knowledge Base (336+ terms indexed). Explore full technical definitions: [https://spidits.com/ai-glossary](https://spidits.com/ai-glossary)*