Deep learning is machine learning with neural networks that have many layers. “Deep” just means the number of layers stacked between input and output: a handful in early networks, hundreds in some modern ones. It is the approach behind today’s language, image, speech and video models.
What sets it apart is who designs the features. In classic machine learning, people decided what the model should look at: word counts for text, edge detectors for images, hand-built measurements for audio. Deep learning feeds in raw data, pixels or tokens, and lets the layers learn their own features. That is representation learning, and the depth is what makes it work.
Why depth helps
The diagram shows the classic picture from image networks. Reading upward, early layers respond to simple patterns such as edges, middle layers combine those into textures and parts, and later layers respond to whole objects. Nobody programs this hierarchy. It emerges from training, because building complex features out of simpler ones is an efficient way to fit the data. The standard textbook frames deep learning as exactly this: a hierarchy of concepts, each defined through simpler ones.
Every layer is trained together, end to end. Backpropagation sends the error signal from the output back through every layer, and gradient descent adjusts all the weights at once.
Why it took off around 2012
The core ideas are decades old. Three things came together to make them practical:
- Data: large labeled sets like ImageNet, and later the text of the web.
- Compute: GPUs, which do the matrix arithmetic of neural networks far faster than general-purpose processors.
- Training fixes: ReLU activations, dropout and better initialization, which let deeper networks train reliably.
The turning point was AlexNet in 2012. A deep convolutional network trained on two GPUs won that year’s ImageNet competition with a top-5 error of 15.3%, against 26.2% for the next-best entry. Deep networks then spread to speech, translation and games, and the transformer extended the same recipe to language at enormous scale.
The catch
Deep models are hungry. They generally need a lot of data and compute, and on small or tabular datasets simpler methods, such as gradient-boosted trees, can do as well. They are also hard to inspect: the learned features are spread across millions or billions of weights, so explaining why a model made a particular decision remains an open research problem.