Reference 4 stops to get here
L1 Regularization
Adding the sum of absolute weights to the loss function, promoting sparsity and feature selection.
Your route here
4 stops · basics first
- Dataset ✓ understood
A collection of data examples used for training, validating, or testing machine learning models.
- Training Data ✓ understood
The examples a model learns its weights from, kept separate from the validation and test data used to check how well it generalizes.
- Overfitting ✓ understood
When a model fits its training data too closely, noise included, so it scores well on examples it has seen and poorly on new ones.
- Regularization ✓ understood
Any change to training meant to make a model generalize better rather than fit its training data more closely, such as weight penalties, dropout or early stopping.
- L1 Regularization · you are here ✓ understood
Where it sits
Explore nearby
Foundations Linear Regression Predicting a number as a weighted sum of input features, with the weights chosen to minimize the squared error on training data. Training L2 Regularization Adding the sum of squared weights to the loss function, penalizing large weights and improving generalization. Foundations Feature Selection Choosing the most relevant features from available data to reduce dimensionality and improve model performance. Training Weight Decay A regularization technique that shrinks weights toward zero during optimization. Equivalent to L2 regularization in standard SGD, but differs when using adaptive optimizers like Adam. Shipping AI Pruning Removing unnecessary weights or neurons from a trained model to reduce size and computation while maintaining performance.