Attention lets a model weight relevant tokens when building representations — each token looks at every other token in the sequence and decides which ones matter most for understanding it.

Attention helps models connect distant words and focus on important context, which is central to transformer architectures. A classic example: in "The trophy does not fit in the suitcase because it is too large," the word "it" refers to "trophy," not "suitcase." Self-attention lets the model connect "it" to the correct earlier word even across many intervening tokens, by comparing every token against every other token rather than only processing left-to-right the way older sequence models did. That's the core reason transformers resolve pronoun references and other long-range dependencies so much better than RNNs.