Optimization

SGD, momentum, RMSProp, Adam — what each one fixes over plain gradient descent

Why

Backprop only computes gradients — the optimizer decides how to use them to update weights. Picking the wrong optimizer or learning rate is the single most common reason a model "doesn't train," and understanding the failure modes each optimizer was designed to fix matters more than knowing their names.

Intuition

Plain SGD takes a step directly opposite the current gradient — like walking downhill by only looking at the slope under your feet, which oscillates badly in narrow valleys. Momentum adds inertia so you keep rolling in a consistent direction instead of zig-zagging. RMSProp gives each parameter its own adaptive step size based on how large its recent gradients have been. Adam combines both ideas: it remembers direction (momentum) and scales per-parameter (adaptive rate).

Explanation
SGD and the momentum fix
SGD: θ = θ - η·∇L(θ) problem: same step size in every direction → oscillates in narrow/curved loss valleys, slow to escape shallow regions Momentum: v = β·v + ∇L(θ) (β ≈ 0.9, exponential moving average) θ = θ - η·v fix: accumulates a "velocity" — directions that keep pointing the same way build up speed, oscillating directions cancel out
RMSProp — per-parameter adaptive rate
RMSProp: s = β·s + (1-β)·(∇L)² (running average of squared gradients) θ = θ - η·∇L / (√s + ε) fix: divides each parameter's step by the magnitude of its own recent gradients — parameters with large/noisy gradients get smaller steps, parameters with small/stable gradients get relatively larger steps

This matters when different parameters need very different effective learning rates — common in deep nets where early and late layers see very different gradient scales.

Adam — momentum + adaptive rate combined
m = β1·m + (1-β1)·∇L first moment (mean, like momentum) v = β2·v + (1-β2)·(∇L)² second moment (variance, like RMSProp) m̂ = m / (1-β1^t) bias-corrected (m,v start at 0, biased early on) v̂ = v / (1-β2^t) θ = θ - η·m̂ / (√v̂ + ε) defaults: β1=0.9, β2=0.999, ε=1e-8

Adam is the default choice for most deep learning because it needs little tuning and adapts per-parameter — but plain SGD with momentum still generalizes better for some vision tasks. Neither is strictly better; the tradeoff is "Adam converges faster, SGD+momentum sometimes generalizes better."

Learning rate schedules and warmup
Step decay: halve η every N epochs Cosine decay: η follows a cosine curve down to ~0 over training Warmup: start η near 0, ramp up linearly for the first few hundred/thousand steps, then decay

Warmup exists because early in training, weights are random and gradients are large/noisy — a full-size learning rate can cause an unstable early update the model never recovers from, especially with Adam's adaptive denominator or in transformers with layer norm. Decay later in training lets the model settle into a sharper minimum instead of oscillating around it.