Faithfulness vs relevance in RAG evaluation?medium

Type
conceptual
Topic
faithfulness-vs-relevance-in-rag-evaluation
Frequency
common
Tags
rag, faithfulness, relevance, evaluation
Answer

Faithfulness: does the generated answer contain only information supported by the retrieved context?

Explanation

Faithfulness: does the generated answer contain only information supported by the retrieved context? Measures hallucination. Relevance: is the answer responsive to the user's question? A response can be faithful (all claims grounded) but irrelevant (answers a different question). Measure faithfulness with NLI or LLM-judge checking claim-by-claim. Measure relevance with semantic similarity between query and answer.

Follow-upWhen would you choose one approach over the other?