Gradient descent iteratively updates weights by subtracting the gradient of the loss scaled by the learning rate: w ← w − α∇L(w). Calculus provides the gradient; the algorithm does the iterative update.

At each step: compute the loss on the current batch, compute ∂L/∂w for each weight via backpropagation, update each weight. The learning rate α controls step size. Variants: batch GD (full dataset per step), stochastic GD (one sample), mini-batch GD (batch of 32-512). Adam and RMSprop adapt the learning rate per weight using gradient history - converging faster than vanilla SGD in most cases.