Earlier NLP represented text with sparse symbolic features like one-hot encoding, bag of words, TF-IDF, and n-grams, but those representations don't capture meaning — every pair of different words is equally distant, so a one-hot model treats "king" and "queen" as no more related than "king" and "apple." Dense embeddings were introduced to fix that: a continuous vector space where similar-meaning words end up near each other.

In a one-hot vocabulary of [king, queen, apple, doctor, hospital], each word is a vector with a single 1 and the rest 0s, so any two different words have the exact same distance from each other — there's no way to encode that king ≈ queen or doctor ≈ hospital while apple sits far from both. That flatness is a real limitation for any model trying to generalize: it has to treat every word pair as equally unrelated regardless of actual meaning. Embeddings solve this by replacing sparse, meaning-blind vectors with dense ones where geometric closeness reflects semantic closeness, which is what makes it possible for a neural network to generalize across related words instead of memorizing each one independently.