Standard 9 stops to get here · leads to 2

Residual Connection

Skip connections that allow gradients to flow directly through a network, enabling training of very deep networks (ResNet).

Your route here

9 stops · basics first
  1. Machine Learning ✓ understood

    Building systems that learn patterns from data instead of following hand-written rules, getting better at a task as they see more examples.

  2. Neural Network ✓ understood

    A computational model inspired by biological neural networks, consisting of interconnected nodes (neurons) organized in layers that process information through weighted connections.

  3. Activation Function ✓ understood

    A non-linear function applied to neuron outputs that introduces non-linearity, enabling networks to learn complex patterns.

  4. Dataset ✓ understood

    A collection of data examples used for training, validating, or testing machine learning models.

  5. Training ✓ understood

    The process of fitting a model to data by repeatedly measuring how wrong its outputs are and adjusting its parameters to reduce that error.

  6. Loss Function ✓ understood

    A function that scores how wrong a model's prediction is as a single number, which training then works to make as small as possible.

  7. Gradient Descent ✓ understood

    An optimization method that repeatedly moves a model's parameters a small step in the direction that most reduces the loss.

  8. Backpropagation ✓ understood

    The algorithm for computing gradients of the loss with respect to network weights, enabling training through gradient descent.

  9. Vanishing Gradient ✓ understood

    A problem where gradients become extremely small during backpropagation, preventing deep networks from learning effectively.

  10. Residual Connection · you are here ✓ understood

Where it sits

Before this

Neural Network Vanishing Gradient
Residual Connection

Leads to

ResNet Neural ODE

Explore nearby

Vision & Multimodal ResNet Residual Network - a CNN architecture using skip connections to enable training of very deep networks (up to 1000+ layers). Neural Networks Skip Connection Direct connections bypassing one or more layers, helping gradient flow and enabling deeper networks. Language & LLMs Transformer A neural network architecture, introduced in 2017, built from stacked self-attention and feed-forward layers; the basis of nearly every modern large language model. Neural Networks Batch Normalization A technique that normalizes layer inputs to stabilize and accelerate training by reducing internal covariate shift.

In the research

All papers →

2 papers that build on Residual Connection .