An embedding is a dense vector in ℝⁿ that represents an entity (word, image, user) in a continuous space where geometric relationships encode semantic meaning.

The famous word2vec result "king − man + woman ≈ queen" is a vector arithmetic result - meaning is encoded as direction and offset in the embedding space. Embedding layers are weight matrices that perform a lookup (sparse one-hot × dense matrix = the row corresponding to that index). Similarity between embeddings is measured with cosine similarity or dot product. The entire retrieval step in RAG is a nearest-neighbor search over an embedding space - a linear algebra operation at scale.