- GPU ✓ understood
Graphics Processing Unit - hardware accelerator with thousands of cores, essential for parallel computation in deep learning.
- Dataset ✓ understood
A collection of data examples used for training, validating, or testing machine learning models.
- Training ✓ understood
The process of fitting a model to data by repeatedly measuring how wrong its outputs are and adjusting its parameters to reduce that error.
- Distributed Training ✓ understood
Training one model across many GPUs or machines at once, by splitting the data, the model, or both, and keeping every copy in sync.
- Data Parallelism ✓ understood
Replicating the model across devices, each processing different data batches.
- Loss Function ✓ understood
A function that scores how wrong a model's prediction is as a single number, which training then works to make as small as possible.
- Gradient Descent ✓ understood
An optimization method that repeatedly moves a model's parameters a small step in the direction that most reduces the loss.
- Learning Rate ✓ understood
The hyperparameter that sets how big a step gradient descent takes on each update; too high makes training unstable, too low makes it crawl.
- Optimizer ✓ understood
The algorithm that turns gradients into weight updates during training, deciding how far and in which direction each parameter moves at every step.
- Momentum ✓ understood
An optimization technique that accelerates gradient descent by accumulating past gradients, helping escape local minima.
- Adam Optimizer ✓ understood
An adaptive learning rate optimization algorithm combining momentum and RMSprop, widely used for training neural networks.
- ZeRO · you are here ✓ understood