A saddle point is where the gradient is zero but it is neither a local minimum nor maximum - the function curves up in some directions and down in others.

Gradient descent stalls near saddle points because the gradient approaches zero. In high-dimensional neural network loss surfaces, saddle points are far more common than bad local minima. SGD noise and adaptive optimizers (Adam's momentum) help escape saddle points. The Hessian at a saddle point has both positive and negative eigenvalues - confirming the mixed curvature. Recognizing saddle points distinguishes convergence issues from architecture or data problems.