Foundation model vs fine-tuned — when to fine-tune vs prompt-engineer?hard

Type
conceptual
Topic
foundation-model-vs-fine-tuned-when-to-fine-tune-vs-prompt
Frequency
common
Tags
llms, foundation, model, fine, tuned, when
Answer

Foundation model: pretrained, general-purpose. Fine-tuning adapts weights with labeled data.

Explanation

Foundation model: pretrained, general-purpose. Fine-tuning adapts weights with labeled data. Prompting steers without changing weights. Start with prompting — cheaper, faster, no data needed. Fine-tune when prompting hits a quality ceiling, you have 100s+ examples, and the task needs consistent format/style. a document extraction pipeline achieved 98% accuracy with prompt engineering alone — fine-tuning was unnecessary.

Follow-upWhen would you choose one approach over the other?