A context window is everything a language model can see while producing its next token. Your system prompt, the conversation so far, pasted documents, tool results and the model’s own reply all have to fit inside one fixed budget, measured in tokens. Anthropic’s documentation describes it as the model’s “working memory”, as opposed to the much larger body of text it was trained on.
Nothing outside the window exists for the model. It has no memory between requests. A chat app feels continuous because it resends the whole history every turn, and when that history gets too long, something has to be dropped or summarized.
How big
Sizes have grown fast. GPT-3, in 2020, had a context window of 2,048 tokens, enough for a few pages of text. As of 2026, Anthropic lists current Claude models at up to 1 million tokens, room for a sizeable codebase. The diagram shows why a big number still runs out: the input and the output draw from the same budget, so a prompt that fills the window leaves no room for an answer.
Why there’s a limit
A transformer uses attention to let every token look at every other token. For a sequence of n tokens, that’s on the order of n² comparisons per layer, so doubling the context roughly quadruples the attention work. Serving long contexts also means holding cached state for every token in memory. Techniques like FlashAttention make long windows practical, and positional encodings have to be designed to cope with lengths the model rarely saw in training.
The catch
A bigger window doesn’t mean the model uses all of it well. In a 2023 study, models answered questions best when the relevant passage sat near the start or end of the input, and noticeably worse when it was buried in the middle. Anthropic’s docs call the general decline with length “context rot”. Longer inputs also cost more, since you pay for every token on every call.
So the skill is choosing what goes in, not just fitting more. Retrieval-augmented generation fetches only the passages that matter. Agent systems hand subtasks to helpers with fresh, smaller contexts. The Duel lets you watch a single model’s window overflow and see what splitting the work saves.