Reference 2 stops to get here · leads to 3

Special Token

Reserved tokens with special meanings like [CLS], [SEP], [MASK], [PAD] used in various model architectures.

Your route here

2 stops · basics first
  1. Token ✓ understood

    The unit of text a language model reads and writes: a word, part of a word, or a character, drawn from the model's fixed vocabulary.

  2. Tokenization ✓ understood

    Splitting text into tokens, usually subword pieces, and mapping each to an integer ID so a language model can process it.

  3. Special Token · you are here ✓ understood

Reserved tokens with special meanings like [CLS], [SEP], [MASK], [PAD] used in various model architectures.

This concept is essential for understanding large language models and forms a key part of modern AI systems.

  • Token
  • BERT
  • Tokenization

Where it sits

Before this

Tokenization
Special Token

Explore nearby