A norm measures the "size" or "length" of a vector. L1 norm: sum of absolute values. L2 norm: square root of sum of squares (Euclidean length). L∞ norm: maximum absolute value.

L2 regularization (Ridge) penalizes the L2 norm of weights - shrinks all weights proportionally. L1 regularization (Lasso) penalizes the L1 norm - drives some weights to exactly zero, producing sparse models. Gradient clipping clips the L2 norm of the gradient to prevent exploding gradients in RNNs and transformers. Normalizing embeddings to unit L2 norm makes cosine similarity equivalent to the dot product (and enables faster FAISS searches).