BERT: encoder-only, bidirectional, pretrained with Masked LM + Next Sentence Prediction.

BERT: encoder-only, bidirectional, pretrained with Masked LM + Next Sentence Prediction. Sees full context — good for classification/extraction. GPT: decoder-only, autoregressive, pretrained with causal LM (predict next token). Good for generation. For a document extraction pipeline extraction, BERT-style is better; for generative summarization (a fund document processing system), GPT-style is better.