Walk me through backpropagation from scratch.medium
Answer
Forward pass: compute activations layer by layer, get a loss.
Explanation
Forward pass: compute activations layer by layer, get a loss. Backward pass: use chain rule to compute gradient of loss w.r.t. each weight. For weight W in layer l: ∂L/∂W = ∂L/∂output × ∂output/∂W. Gradients flow backward through activation derivatives (ReLU → 1 if x>0 else 0, sigmoid → σ(1-σ)). Weights updated: W = W - lr × ∂L/∂W.
Follow-upWhat tradeoffs did you consider in that implementation?