Relevance measures whether the answer addresses the user's question; faithfulness measures whether the answer is supported by the retrieved context. In RAG, an answer can be relevant but unfaithful if it answers the question using invented information, or faithful but irrelevant if it quotes correct context that doesn't actually answer the question.

Faithfulness: does the generated answer contain only information supported by the retrieved context? Measures hallucination. Relevance: is the answer responsive to the user's question? There are four possible cases. Relevant and faithful is the best case — the answer addresses the question and is grounded in the source. Relevant but unfaithful is the dangerous case — the answer sounds like it addresses the question but states something the context doesn't support (e.g. context says a 7-day refund window, the answer says 30 days); this is a hallucination even though it reads as useful. Faithful but irrelevant is when the answer is accurate to the document but doesn't answer what was asked (e.g. answering 'support is available by email' to a refund-period question). Neither relevant nor faithful is the worst case — unsupported and unhelpful. Measure faithfulness with NLI or an LLM-judge checking claim-by-claim against the context; measure relevance with semantic similarity between query and answer, or an LLM-judge scoring whether the question was actually addressed. A strong RAG system needs both — useful to the user and grounded in the retrieved sources — since optimizing for one alone lets the other quietly fail.