ELMo introduced contextual word embeddings using bidirectional LSTM language models — a genuine shift from Word2Vec's one-word-one-vector approach to a vector that's a function of the entire sentence. "Bank" in "I deposited money in the bank" and "I sat near the river bank" gets different ELMo representations, solving the polysemy problem Word2Vec couldn't.

ELMo reads text in both directions — a forward LSTM (I → deposited → money → in → the → bank) and a backward LSTM (bank → the → in → money → deposited → I) — then combines both directions' representations, letting the model use both left and right context to disambiguate a word. For "The bat flew at night" versus "He swung the bat," ELMo produces different representations depending on whether the sentence is about the animal or the sporting object. Before ELMo, an embedding was a fixed lookup vector for a word; after ELMo, an embedding became a function of the entire sentence. That reframing was a genuine bridge on the path to modern LLMs. ELMo still inherited LSTM's limitations though — sequential computation, weaker parallelization, and difficulty scaling to very long context — which is exactly what Transformers and self-attention solved next.