A gradient is the vector of partial derivatives of a multi-variable function - it points in the direction of steepest increase. Moving opposite the gradient decreases the function (gradient descent).

For a loss function L(w₁, w₂, ..., wₙ), the gradient ∇L = [∂L/∂w₁, ∂L/∂w₂, ..., ∂L/∂wₙ]. Each element tells you how much the loss changes with a small change in that weight. The gradient has magnitude (how steep) and direction (which way). Optimization: take a small step in the negative gradient direction -> reduce loss. The gradient is zero at any critical point (minimum, maximum, or saddle point).