Text is tokenized, each token ID is mapped to a fixed initial embedding vector, and that embedding is only the starting point — once it passes through the transformer's self-attention layers, it becomes a contextual representation that shifts based on surrounding words.

The input flow is text → tokens → token IDs → embedding vectors → transformer layers. Tokens that appear in similar contexts end up with similar initial vectors (doctor ≈ physician, car ≈ vehicle), but that initial embedding is the same regardless of sentence. The classic example: "I deposited money in the bank" and "I sat near the river bank" start with the identical initial embedding for "bank," but after self-attention pulls in context from the surrounding words ("deposited," "money" versus "sat," "river"), the resulting representations diverge to reflect the two different meanings. This is the core reason transformer-based models handle polysemy (words with multiple meanings) far better than static embedding methods like Word2Vec, which assign one fixed vector per word regardless of context.