The atomic unit of data in ML
Every piece of data in ML is a vector. A user is a vector of features. A word is a vector of 768 floats. An image is a vector of pixel values. A model's hidden state is a vector. Before you can understand attention, embeddings, similarity search, or neural networks, you need to know what a vector is and what you can do with it.
Without this, the phrase "cosine similarity between two embeddings" is gibberish. With it, it's immediately obvious what's happening.
A vector is just a list of numbers with a direction. Think of it as an arrow in space. A 2D vector [3, 4] is an arrow that goes 3 steps right and 4 steps up. A 768-dimensional vector is the same idea — just an arrow in a space you can't visualize.
A vector is an ordered list of numbers: v = [v₁, v₂, ..., vₙ]. The number of elements is the dimension. Vectors can be written as a column vector (n×1 — default in ML literature) or a row vector (1×n). This distinction matters when multiplying with matrices — shape errors are almost always a row/column confusion.
The magnitude is how far the arrow reaches from the origin — square root of the sum of squared components:
For v = [3, 4]: ||v|| = √(9 + 16) = 5. The L1 norm is the sum of absolute values. Both appear as regularization penalties — Ridge = L2, Lasso = L1.
A unit vector has magnitude = 1. To normalize: û = v / ||v||. When comparing word embeddings you normalize first so similarity is purely about direction, not magnitude.