Faithfulness vs relevance in RAG evaluation?medium
Answer
Faithfulness: does the generated answer contain only information supported by the retrieved context?
Explanation
Faithfulness: does the generated answer contain only information supported by the retrieved context? Measures hallucination. Relevance: is the answer responsive to the user's question? A response can be faithful (all claims grounded) but irrelevant (answers a different question). Measure faithfulness with NLI or LLM-judge checking claim-by-claim. Measure relevance with semantic similarity between query and answer.
Follow-upWhen would you choose one approach over the other?