Tokenization splits text into units such as words, subwords, or characters — the model never reads raw characters directly, only these token IDs. That choice quietly affects context length, API cost, multilingual performance, and even why a model can fail at simple tasks like counting letters.

Tokenization converts raw text into model-readable pieces. Modern models often use subword tokenization to handle rare words and multiple languages, so a sentence like "ChatGPT is useful." might split into pieces like Chat | G | PT | is | useful | . rather than clean whole words. That has real practical impact: one word can become multiple tokens, so a "4K token" window can hold noticeably fewer than 4K words; API pricing is per token, so longer tokenization directly raises cost; non-English text often tokenizes into more tokens per character, effectively shrinking usable context for those languages; and code identifiers, indentation, and symbols may each land as separate tokens, shaping how the model reasons about code structure. It's also why counting or spelling inside a word is unreliable — a word like "strawberry" may be represented as one or two subword tokens rather than ten separate letters, so the model has no direct way to enumerate characters and instead predicts a plausible-sounding count based on pattern recognition.