What is Batch Size?

Detailed Deep Dive

Batch size is a critical training hyperparameter that defines the number of training samples processed in one forward and backward pass before the model's internal weights are updated. A smaller batch size provides a noisier gradient estimate, which can help escape local minima, but is less computationally efficient. A larger batch size offers smoother gradients and better utilizes parallel GPU processing power but can sometimes result in poorer generalization.