Turning text into numbers a computer can compare — the bridge that makes semantic search possible
Computers can't directly compare "meaning" — they can only do math on numbers. Embeddings are the bridge: convert meaning into numbers, then use simple math like cosine similarity to measure how close two pieces of text are in meaning. Without this, there is no way to find relevant chunks by anything other than exact keyword matching.
An embedding model takes a piece of text and converts it into a vector — a list of numbers (e.g., 384, 768, or 3072 of them, depending on the model) representing its meaning in a mathematical space. Texts with similar meaning end up with vectors that are close to each other, even when they don't share any of the same words: "The dog is running in the park" and "A canine sprints across the field" land close together, because embedding models capture semantic meaning, not just keyword overlap.
Cosine similarity between two vectors A and B is the cosine of the angle between them:
Most embedding models normalize every output vector to length 1 (||A|| = 1) automatically. Once every vector has length 1, cosine similarity and the raw dot product become the same number — which is why some vector databases default to the (cheaper) dot product internally while calling it "cosine" in the API: the normalization already happened when the vectors were created.
Embedding models are benchmarked on MTEB (the Massive Text Embedding Benchmark), which scores models across retrieval, classification, clustering, and other tasks — retrieval performance specifically is the number that matters for RAG, and it doesn't always track a model's overall MTEB rank.
text-embedding-3-small/large, Cohere's embed-v3, Google's text-embedding-004 — no infrastructure to run, billed per token, easy to start with. Every document embedded through an API call means your entire corpus's text passes through that provider.bge-large-en, bge-m3), E5, and Sentence-Transformers models (all-MiniLM-L6-v2 for lightweight use) can be self-hosted, inspected, and fine-tuned, at the cost of running your own inference infrastructure. Frequently competitive with or ahead of proprietary models on MTEB retrieval leaderboards.Dimension count is a real trade-off, not just a technical detail: a 3072-dimension vector carries more information but costs roughly 4x the storage and search compute of a 768-dimension one for the same number of chunks — worth checking against actual retrieval quality before defaulting to the largest available model.
Everything above describes dense embeddings — every one of the vector's hundreds or thousands of dimensions holds a non-zero, learned value, and meaning is distributed across all of them. Sparse embeddings (e.g., SPLADE) instead produce a vector nearly as large as the vocabulary itself, where most values are exactly zero and the non-zero ones roughly correspond to specific important terms — conceptually a learned, neural upgrade over classic keyword matching rather than a pure meaning-based representation. Sparse vectors keep some of BM25's exact-term strength (see Search Types) while still benefiting from learned term importance, and are increasingly used as one leg of a hybrid search setup alongside dense embeddings.
Named after Russian nesting dolls. Some embedding models (like OpenAI's text-embedding-3-large, or nomic-embed-text) are trained so a vector can be truncated — e.g., cutting a 3072-dimension vector down to its first 256 dimensions — and it still works reasonably well, just slightly less precisely, instead of becoming garbage. Smaller truncated vectors mean less storage and faster search, without training a separate small model.
Whatever embedding model indexes your documents must also embed the user's query at search time. Vectors from two different embedding models — or even two versions of the same model — live in totally different, incompatible mathematical spaces; comparing them is meaningless, like comparing distances measured in miles against kilometers without converting.
This has a practical consequence teams often miss: switching embedding models is not a config change, it's a full data migration. The entire document set has to be re-embedded from scratch, and until that finishes, you either run two vector collections side by side (old model serving queries against the old index, new model building the new index) or accept a period where retrieval quality is degraded mid-migration. Track the embedding model (and version) as metadata on the index itself, not just in application config, so this is never ambiguous during an incident.
Models like CLIP embed images and text into the same vector space, so a text query can retrieve a relevant image (or vice versa) via ordinary cosine similarity — no separate image-captioning step required. This extends RAG to corpora that are partly or entirely visual: product catalogs, diagrams, scanned figures, or screenshots, where the "chunk" being retrieved is an image rather than a paragraph of text.
General-purpose embedding models are trained on broad internet text. If a domain has very specific jargon — insurance policy terms, medical terminology, internal company acronyms — a general model might not cluster those concepts as precisely. An embedding model can be further trained on your own (query, correct-matching-document) pairs using contrastive learning, which pulls matching pairs closer together in vector space and pushes non-matching pairs apart. This is an advanced, optional step — most projects work fine with an off-the-shelf embedding model, and it's usually worth trying prompt-level fixes (better chunking, hybrid search) before investing in fine-tuning an embedding model.