A derivative measures the instantaneous rate of change of a function at a point - the slope of the tangent line. For f(x), the derivative f'(x) tells you how fast f changes as x changes.

In ML, derivatives appear as gradients: the derivative of a loss function with respect to model weights tells you which direction to adjust each weight to reduce the loss. If f'(x) > 0, increasing x increases f. If f'(x) < 0, increasing x decreases f. At a minimum, f'(x) = 0. The second derivative tells you whether it is a minimum (positive), maximum (negative), or saddle point (zero).