Derivatives when you have multiple inputs — like millions of weights
A neural network has millions of parameters. The loss function depends on all of them simultaneously. A regular derivative only handles one variable. Partial Derivatives extend this: they let you ask "how does the loss change with respect to this one specific weight, while holding all others fixed?" This is exactly what backprop computes — a partial derivative for every single weight in the network.
Imagine a landscape where your position is described by two coordinates (x, y) and your altitude is f(x, y). A partial derivative ∂f/∂x asks: if I take one step in the x-direction only (keeping y frozen), how much does my altitude change? ∂f/∂y asks the same for y.
In ML: x and y are two weights. f is the loss. Partial Derivatives tell you the slope in each weight's direction independently.
Treat all other variables as constants, then differentiate normally with respect to the target variable.
The notation ∂ (curly d) distinguishes partial from total derivatives. Read ∂f/∂wᵢ as "the partial derivative of f with respect to wᵢ."
In a simple linear model: ŷ = w·x + b, loss = MSE = (y - ŷ)²
These two partial derivatives tell you exactly how to update w and b to reduce the loss. This is the update rule for linear regression gradient descent.
A deep network has parameters w₁, w₂, ..., wₙ (millions of them). Backprop computes ∂L/∂wᵢ for every single weight. Each tells you: nudge this weight in this direction by this much to reduce the loss. The collection of all these partial derivatives is the gradient — covered next.