Reference 2 stops to get here

Stop Words

Common words (the, is, at) often removed in NLP preprocessing as they carry little semantic meaning.

Your route here

2 stops · basics first
  1. Token ✓ understood

    The unit of text a language model reads and writes: a word, part of a word, or a character, drawn from the model's fixed vocabulary.

  2. Tokenization ✓ understood

    Splitting text into tokens, usually subword pieces, and mapping each to an integer ID so a language model can process it.

  3. Stop Words · you are here ✓ understood

Where it sits

Before this

Tokenization
Stop Words

Leads to

Nothing yet: a destination in its own right.

Explore nearby