Standard 7 stops to get here

Cosine Similarity

How closely two vectors point in the same direction, from 1 (same direction) to -1 (opposite): the standard way to compare embeddings.

Your route here

7 stops · basics first
  1. Dataset ✓ understood

    A collection of data examples used for training, validating, or testing machine learning models.

  2. Feature ✓ understood

    A single measurable property of an example, such as a house's floor area or how many links an email contains, used as an input to a model.

  3. Machine Learning ✓ understood

    Building systems that learn patterns from data instead of following hand-written rules, getting better at a task as they see more examples.

  4. Neural Network ✓ understood

    A model built from layers of simple units, each taking a weighted sum of its inputs and applying a nonlinearity; training adjusts the weights so the whole stack maps inputs to useful outputs.

  5. Deep Learning ✓ understood

    A subset of machine learning that uses neural networks with multiple layers (deep neural networks) to learn hierarchical representations of data.

  6. Representation Learning ✓ understood

    Letting a model learn its own internal description of the data, such as embeddings or layered visual features, instead of relying on features designed by hand.

  7. Embedding ✓ understood

    A list of numbers (a vector) that represents a word, sentence, image or other item, learned so that similar items end up close together.

  8. Cosine Similarity · you are here ✓ understood

Cosine similarity measures the angle between two vectors and ignores their length. Take the dot product of the two vectors and divide by the product of their lengths: the result is the cosine of the angle between them. Vectors pointing the same way score 1, perpendicular vectors score 0, and opposite vectors score -1.

Why angle, not distance

Information retrieval made the cosine standard. Represent each document as a vector of word counts, and a long document and a short one about the same topic point the same way while sitting far apart, simply because one has more words. Manning, Raghavan and Schütze’s textbook gives exactly this reason for using the cosine: it compensates for document length. Measuring the angle compares what two things are about rather than how much of it there is.

Where you meet it

The same logic carries over to embeddings. Models place words, sentences and images in a vector space where related items point in similar directions, and cosine similarity is the usual score for how related two of them are. The word2vec paper answered analogy questions by finding the word vector closest to a computed one under cosine distance (the paper).

Today it is the default comparison in semantic search, in vector databases, and in the retrieval step of RAG: embed the question, then return the stored passages with the highest cosine similarity. Contrastive models such as CLIP are trained so that matching pairs score high on it.

A practical note: if every vector is normalized to length 1 first, cosine similarity is just the dot product, which is why many systems normalize embeddings once and then use fast dot-product search. And a high score means the model placed two items close together, not that they are true, relevant or interchangeable.

Where it sits

Before this

Embedding
Cosine Similarity

Leads to

Nothing yet: a destination in its own right.

Explore nearby

Language & LLMs Semantic Similarity Measuring how similar two pieces of text are in meaning, often using embedding-based distance metrics. Language & LLMs Vector Database A database optimized for storing and searching high-dimensional vectors (embeddings), enabling semantic search and RAG. Language & LLMs Word2Vec A technique for learning word embeddings that capture semantic relationships (Skip-gram and CBOW models). Language & LLMs Retrieval-Augmented Generation Fetching relevant documents at question time and adding them to an LLM's prompt, so answers can draw on current, checkable sources without retraining. Training Contrastive Learning A self-supervised learning approach that learns representations by contrasting similar and dissimilar examples. Vision & Multimodal CLIP Contrastive Language-Image Pre-training - a model jointly trained on images and text, enabling zero-shot image classification.

Sources

  1. Christopher D. Manning, Prabhakar Raghavan and Hinrich Schütze, Introduction to Information Retrieval . Cambridge University Press, 2008; section 6.3.1, Dot products
  2. Tomas Mikolov et al., "Efficient Estimation of Word Representations in Vector Space" . 2013; word analogies answered by the nearest vector under cosine distance