Pretraining vs fine-tuning vs prompt-based learning — differences?hard
Answer
Pretraining: train on massive corpus to learn general representations (expensive, done once).
Explanation
Pretraining: train on massive corpus to learn general representations (expensive, done once). Fine-tuning: update weights on task-specific labeled data — strong performance, needs labels. Prompt-based: craft inputs to steer a frozen LLM — zero/few-shot, no gradient updates. In production: prompt engineering first (fast, cheap), fine-tune if quality insufficient, pretrain only if domain is radically out-of-distribution.
Follow-upWhen would you choose one approach over the other?