Standard 5 stops to get here

Mini-Batch Gradient Descent

Computing gradients on small batches of data, balancing SGD's noise with full-batch GD's stability.

Your route here

5 stops · basics first
  1. Dataset ✓ understood

    A collection of data examples used for training, validating, or testing machine learning models.

  2. Training ✓ understood

    The process of fitting a model to data by repeatedly measuring how wrong its outputs are and adjusting its parameters to reduce that error.

  3. Loss Function ✓ understood

    A function that scores how wrong a model's prediction is as a single number, which training then works to make as small as possible.

  4. Gradient Descent ✓ understood

    An optimization method that repeatedly moves a model's parameters a small step in the direction that most reduces the loss.

  5. Batch Size ✓ understood

    The number of training examples processed together in one forward/backward pass.

  6. Mini-Batch Gradient Descent · you are here ✓ understood

Computing gradients on small batches of data, balancing SGD’s noise with full-batch GD’s stability.

This concept is essential for understanding training & optimization and forms a key part of modern AI systems.

  • SGD
  • Gradient Descent
  • Batch Size

Where it sits

Before this

Gradient Descent Batch Size
Mini-Batch Gradient Descent

Leads to

Nothing yet: a destination in its own right.

Explore nearby

Training Stochastic Gradient Descent A variant of gradient descent that updates parameters using gradients computed on a single random training example at a time (though often used to refer to mini-batch gradient descent). Training Batch Gradient Descent Computing gradients using the entire dataset, providing stable but slow updates. Training Epoch One complete pass through the entire training dataset during the training process. Training Gradient Accumulation Summing gradients over multiple batches before updating, simulating larger effective batch sizes.