What is the chain rule?medium
Answer
The chain rule computes the derivative of a composite function: if y = f(g(x)), then dy/dx = f'(g(x)) dot g'(x). The derivative of the outer function times the derivative of the inner function.
Explanation
In neural networks, the output is a composition of many functions (layers). Backpropagation applies the chain rule repeatedly, propagating gradients backward through each layer: the gradient at layer L = gradient at layer L+1 × local derivative at layer L. Without the chain rule, training deep networks would be mathematically intractable.
Follow-upWhy does the chain rule make deep networks trainable?