Self-attention lets every token in a sequence compare itself against every other token in the same sequence to build a context-aware representation.

For a word like “bank,” self-attention lets the model weigh nearby words like “river” or “deposited” to decide which meaning applies, updating each token's representation based on the tokens most relevant to it.