Why Hallucinate

LLMs interview question · Medium difficulty

Why do LLMs hallucinate?medium

Status
✓
Type
conceptual
Topic
why-hallucinate
Frequency
common
Tags
hallucination, next-token-prediction, training-objective
Answer

LLMs are trained to predict the most likely next token, not to verify truth against a source. The objective is a fluent, coherent continuation of the prompt, not a fact-check — so when the model has incomplete, conflicting, or outdated information, it can still generate a confident-looking but wrong answer.

Explanation

A model does not "know" facts the way a database does; it generates answers based on patterns learned from training data, and fluency and accuracy are separate properties — a model can be very fluent while being factually wrong. Main causes: the training objective optimizes for what answers usually look like, not what is true; the model has no built-in source of truth unless connected to tools, documents, or RAG; knowledge gaps get filled with plausible-sounding patterns; training data itself can be outdated or contradictory and get blended together; fluency makes wrong information look convincing; prompt pressure can push the model to answer rather than admit it doesn't know; and long-context conversations can cause it to mix up or lose track of details. Hallucination is most likely on niche or low-coverage topics, information past the training cutoff, exact numbers and citations, API/SDK specifics, and long-context conversations where earlier detail was partially lost. For example, asked for an API endpoint it doesn't actually know, a model may generate something that follows a common pattern like GET /api/v1/x/{id}/details — plausible, but possibly nonexistent.

Follow-upWhy can a model be fluent and still be wrong?
Follow-upWhy does hallucination get worse in long-context conversations?
Follow-upHow would you reduce hallucination in a production RAG system?