Matrix multiplication

The single most important operation in deep learning

Why

Matrix multiplication is what makes neural networks fast. A forward pass through a linear layer is a matrix multiply. Running inference on a batch of 64 samples is a matrix multiply. The entire efficiency of GPU-accelerated deep learning comes from GPUs being exceptionally good at matrix multiplications. Without understanding this, you can't reason about model efficiency, memory, or batch processing.

Intuition

Matrix multiplication is doing many dot products at once. If A has m rows and B has n columns, A×B produces an m×n matrix where entry (i,j) is the dot product of row i of A with column j of B.

The critical rule: inner dimensions must match. A(m×k) × B(k×n) → C(m×n). The outer dimensions become the result shape.

Explanation

How matrix multiplication works

Each element C[i][j] = dot(row_i of A, col_j of B)

A = [[1, 2], B = [[5, 6], [3, 4]] [7, 8]] C[0][0] = 1×5 + 2×7 = 19 C[0][1] = 1×6 + 2×8 = 22 C[1][0] = 3×5 + 4×7 = 43 C[1][1] = 3×6 + 4×8 = 50 C = [[19, 22], [43, 50]]

Shape rule — the most important thing to memorize

(m × k) · (k × n) → (m × n) ↑___↑ inner dims must match outer dims are result shape

Linear layer: input (batch=32, features=256) × weights (256, 512) → output (32, 512)
Attention scores: Q(seq=10, d=64) × Kᵀ(d=64, seq=10) → scores(10, 10)
Matrix multiply is NOT commutative: A×B ≠ B×A in general

Batch matrix multiply in practice

In practice, you always process batches. If X is (32, 256) and W is (256, 512), then X @ W produces (32, 512) — all 32 forward passes computed simultaneously. This is why GPUs are effective: they parallelize thousands of dot products at once.

In PyTorch: torch.matmul(X, W) or the @ operator. Know the shapes before you call it.

← Previous Next →