During training: randomly zero out activations with probability p, forcing the network to learn redundant representations.

During training: randomly zero out activations with probability p, forcing the network to learn redundant representations. During inference: disabled, activations scaled by (1-p) to maintain expected values (or training uses inverted dropout: scale by 1/(1-p)). In Transformers, dropout applied after attention weights and after FFN layers. Typical p=0.1.