Transformers scaled well with parallel training, large datasets, and huge parameter counts, which made it practical to pretrain massive next-token-prediction models that became modern LLMs.

Self-attention gave transformers strong context modeling, and their parallelizability let researchers train on far more data and compute than RNNs ever could. Pretraining on next-token prediction, followed by fine-tuning and instruction tuning, turned that scale into general-purpose language ability.