It is the maximum number of tokens the model can attend to at once. Despite having billions of parameters, the window stays limited mainly because self-attention cost grows faster than linearly with sequence length, which hits GPU memory, latency, and serving cost.

Inputs, retrieved context, instructions, conversation history, and outputs all consume context, so long tasks need careful context management. It helps to separate two different kinds of "capacity": parameters are what the model learned during training (long-term knowledge), while the context window is what it can actively read right now (working memory) — a model can know a huge amount and still only process a bounded number of tokens per call. The technical reason the window is capped is attention cost: in basic self-attention every token compares itself with every other token, so doubling context length is closer to 4x the attention work, not 2x. That cost shows up as GPU memory pressure, higher latency, and higher serving cost as context grows. Beyond cost, quality can also degrade with very long context — models can miss details buried deep in the prompt or over-focus on irrelevant text — so handling long context well takes deliberate architecture and training choices (positional encoding strategy, long-context training data), not just raising the number.