Transformers are deep networks, and deep networks are hard to train without stabilizing techniques. Residual connections let each layer add a learned change to its input rather than replacing it outright (output = input + transformation(input)), which keeps gradients flowing through many stacked layers. Layer normalization stabilizes the distribution of activations at each layer, independent of batch size. Together, they're what makes training a transformer dozens of layers deep actually tractable.

Without residual connections, gradients have to flow backward through every transformation in every layer unmodified, and in a sufficiently deep stack that signal degrades — residual connections give gradients a direct additive path back to earlier layers, sidestepping that degradation. Layer normalization (as opposed to batch normalization) normalizes across the feature dimension for each individual token, independent of the rest of the batch, which is why transformers use it instead of batch norm — it doesn't break down on variable-length sequences or small batches the way batch norm can. Neither technique changes what the model is capable of representing; they change whether that representation is actually reachable through gradient descent at depth. That's why removing either one from a deep transformer tends to make training unstable or painfully slow, even though the architecture is technically unchanged in terms of capacity.