Landmark 5 stops to get here · leads to 2

Representation Learning

Letting a model learn its own internal description of the data, such as embeddings or layered visual features, instead of relying on features designed by hand.

Your route here

5 stops · basics first
  1. Dataset ✓ understood

    A collection of data examples used for training, validating, or testing machine learning models.

  2. Feature ✓ understood

    A single measurable property of an example, such as a house's floor area or how many links an email contains, used as an input to a model.

  3. Machine Learning ✓ understood

    Building systems that learn patterns from data instead of following hand-written rules, getting better at a task as they see more examples.

  4. Neural Network ✓ understood

    A model built from layers of simple units, each taking a weighted sum of its inputs and applying a nonlinearity; training adjusts the weights so the whole stack maps inputs to useful outputs.

  5. Deep Learning ✓ understood

    A subset of machine learning that uses neural networks with multiple layers (deep neural networks) to learn hierarchical representations of data.

  6. Representation Learning · you are here ✓ understood

Picture it

  1. Learned representation A compact vector a downstream task can use
  2. Objects Deeper layers respond to whole concepts
  3. Textures and parts Middle layers combine edges into motifs
  4. Edges Early layers learn simple local patterns
  5. Raw pixels The input, with no hand-crafted features
Notice that no one designed these features: each layer learns them from data, building more abstract representations on top of simpler ones.

Representation learning is the idea that a model should work out for itself how to describe its input, instead of relying on features a person designed. The “representation” is the set of numbers the model uses internally to stand for an image, a word or a user. A good one makes the eventual task easy: similar things land close together, and the differences that matter are easy to read off.

Before deep learning, much of the effort in a machine learning project went into feature engineering: hand-crafting edge detectors for images or word counts for text. A 2012 review by Bengio, Courville and Vincent argued that a learner’s success depends heavily on its representation, and that the pursuit of AI calls for learning representations from data rather than engineering them.

What gets learned

The diagram follows an image through a trained vision network. Zeiler and Fergus visualized this in 2013: early layers responded to edges and colour patches, middle layers to textures and object parts, and the deepest layers to whole objects such as dog faces. Nobody told the network to build edge detectors. They emerged because they were useful for the task, and each layer built its descriptions from the one below.

Words work the same way. Word2vec (2013) learned vectors for words from raw text, and the resulting space had structure nobody programmed in: the vector for “king”, minus “man”, plus “woman”, lands closest to “queen”. These learned vectors are embeddings, and the space they live in is a latent space.

How representations are learned

  • As a by-product of a supervised task. Train a classifier and its second-to-last layer becomes a general-purpose image description.
  • Without labels. Autoencoders learn to compress and rebuild their input, so the bottleneck has to capture what matters.
  • From the data itself. Self-supervised learning invents a task from raw data, such as predicting a missing word or matching two crops of the same photo. This is how large language and vision models learn representations at scale.

Why it matters

A good representation transfers. Features learned on one large dataset can be reused for a different, smaller task, which is the basis of transfer learning and of every pre-trained foundation model.

The catch is that learned representations are hard to inspect. The numbers work, but what each dimension means is rarely clear, and a representation can quietly encode shortcuts or biases in the data alongside the useful structure.

Where it sits

Before this

Deep Learning Feature
Representation Learning

Leads to

Embedding Information Bottleneck

Explore nearby

Language & LLMs Embedding A list of numbers (a vector) that represents a word, sentence, image or other item, learned so that similar items end up close together. Neural Networks Latent Space A compressed, learned representation space where similar data points are close together, used in autoencoders and VAEs. Training Self-Supervised Learning Training on unlabeled data by making the data supply its own answers: hide or alter part of an input and have the model predict or match it. Neural Networks Autoencoder An unsupervised neural network that learns to compress data into a latent representation and reconstruct it, useful for dimensionality reduction. Foundations Feature Engineering The process of selecting, transforming, and creating input features to improve model performance.

In the research

All papers →

3 papers that build on Representation Learning .

Sources

  1. Bengio, Courville and Vincent, "Representation Learning: A Review and New Perspectives" . 2012; IEEE TPAMI, 2013
  2. Zeiler and Fergus, "Visualizing and Understanding Convolutional Networks" . 2013; ECCV 2014
  3. Mikolov et al., "Efficient Estimation of Word Representations in Vector Space" . 2013 (word2vec)