Singular Value Decomposition factors any matrix A = UsumVᵀ into two orthogonal matrices and a diagonal matrix of singular values. It reveals the inherent structure of the data transformation.

U: left singular vectors (output space directions). sum: singular values (scaling, sorted descending). Vᵀ: right singular vectors (input space directions). Truncated SVD keeps only the top-k singular values and vectors - the best rank-k approximation of A. Applications: PCA (SVD of the data matrix), collaborative filtering (matrix factorization for recommendation), NLP word embeddings (SVD of co-occurrence matrices), and image compression. LoRA uses low-rank decomposition inspired by SVD to reduce fine-tuning parameters.