What is the chain rule?medium

Type
conceptual
Topic
differentiation
Frequency
very common
Tags
differentiation
Answer

The chain rule computes the derivative of a composite function: if y = f(g(x)), then dy/dx = f'(g(x)) dot g'(x). The derivative of the outer function times the derivative of the inner function.

Explanation

In neural networks, the output is a composition of many functions (layers). Backpropagation applies the chain rule repeatedly, propagating gradients backward through each layer: the gradient at layer L = gradient at layer L+1 × local derivative at layer L. Without the chain rule, training deep networks would be mathematically intractable.

Follow-upWhy does the chain rule make deep networks trainable?