What is vanishing gradient?medium

Type
conceptual
Topic
vanishing-gradient
Frequency
common
Tags
vanishing, gradient
Answer

It happens when gradients become too small for early layers to learn.

Explanation

Deep networks can suffer from tiny gradients during backpropagation. Better activations, normalization, residual connections, and initialization help.

Follow-upHow do residual connections reduce this problem?