Modern Architecture

From Code to Transformers

How text becomes numbers, what attention actually does, and why transformers took over AI.

Your progress 0 / 5 concepts

Concepts in This Path

Animated Tutorial

3 related

Embedding

A dense vector representation of discrete data (words, tokens) in continuous space, capturing semantic relationships.

Watch animation

Animated Tutorial

4 related

Tokenization

The process of breaking text into smaller units (tokens) that language models can process, using algorithms like BPE or WordPiece.

Watch animation

Animated Tutorial

4 related

Attention Mechanism

A technique that allows neural networks to focus on relevant parts of the input when producing each output, assigning different weights to different input elements.

Watch animation

Animated Tutorial

5 related

Transformer

A neural network architecture introduced in 'Attention is All You Need' (2017) that relies entirely on self-attention mechanisms, becoming the foundation for modern LLMs.

Watch animation

Animated Tutorial

5 related

Context Window

The maximum number of tokens an LLM can process at once, including both input prompt and generated output. Also called context length.

Watch animation

Ready for more?

After mastering the fundamentals, continue to the next learning path.

All learning paths • Browse full glossary