What is instruction tuning?medium
Answer
Instruction tuning trains a model to follow task instructions more reliably.
Explanation
It uses examples of instructions and desired responses so the model becomes better at conversational and task-oriented behavior than raw next-token prediction alone.
Follow-upHow is instruction tuning different from pretraining?