Reference 2 stops to get here · leads to 4

Special Token

Reserved tokens with special meanings like [CLS], [SEP], [MASK], [PAD] used in various model architectures.

Your route here

2 stops · basics first

Token ✓ understood
The basic unit of text that a language model processes, typically representing a word, subword, or character. Tokens are the fundamental building blocks for LLM input and output.
Tokenization ✓ understood
Splitting text into tokens, usually subword pieces, and mapping each to an integer ID so a language model can process it.
Special Token · you are here ✓ understood

Reserved tokens with special meanings like [CLS], [SEP], [MASK], [PAD] used in various model architectures.

This concept is essential for understanding large language models and forms a key part of modern AI systems.

Token
BERT
Tokenization

Where it sits

Before this

Special Token

Leads to

BOS Token EOS Token Padding Token Stop Token

Explore nearby

Language & LLMs BOS Token Beginning Of Sequence token - marks the start of a sequence in language models. Language & LLMs EOS Token End Of Sequence token - signals when the model has finished generating a complete output. Language & LLMs Padding Token A special token used to make sequences the same length in a batch, typically ignored during computation. Language & LLMs Stop Token A special token signaling the end of generation, causing the model to stop producing more tokens. Language & LLMs BERT Bidirectional Encoder Representations from Transformers - a model that understands context by looking at text from both directions.