Reference 2 stops to get here

Lemmatization

Reducing words to their base or dictionary form (running → run) using linguistic knowledge.

Your route here

2 stops · basics first
  1. Token ✓ understood

    The unit of text a language model reads and writes: a word, part of a word, or a character, drawn from the model's fixed vocabulary.

  2. Tokenization ✓ understood

    Splitting text into tokens, usually subword pieces, and mapping each to an integer ID so a language model can process it.

  3. Lemmatization · you are here ✓ understood

Where it sits

Before this

Tokenization
Lemmatization

Leads to

Nothing yet: a destination in its own right.

Explore nearby