Build a golden dataset of user questions, expected answers, and relevant source chunks, then run the full question-to-retrieval-to-generation path against it offline before deployment, and keep monitoring production signals online after deployment — the two catch different classes of failure.

A golden dataset contains user questions, expected answers, the relevant source documents or chunks, accepted variations, and rejected answers — it's what lets you score the whole pipeline (question → retrieval → context → generation → final answer) rather than just one stage. Offline evaluation runs before deployment: a benchmark test set, regression tests so a change doesn't silently break a previously-working query, prompt comparison, retriever comparison, and embedding model comparison. Online evaluation runs after deployment, where real user behavior surfaces failures a static test set can't: user feedback, thumbs up/down, click-through, escalation rate, correction rate, and production hallucination reports. A practical checklist ties it together — did retrieval fetch the right chunks, were they ranked well, were irrelevant chunks filtered, did the model actually use what it retrieved, is the answer supported and cited correctly, did the system avoid unsupported claims, is the answer useful, and are latency and cost acceptable. The reason both offline and online matter is that offline catches regressions before users see them, while online catches the failure modes — drift, edge cases, adversarial queries — that a fixed golden dataset can't anticipate.