RAG and Evaluation of Generative Systems

1. Why measure retrieval before the quality of the final answer?
2. What does recall@k mean in a set where the supporting passage is annotated?
3. At which level should the citations of a generated answer be verified?
4. Why include unanswerable questions in the evaluation set?
5. Why apply access rights before document search?

Answers. 1. An end-to-end score does not localize the failure; without a supporting passage in context, the generation cannot be correctly blamed. 2. The proportion of questions for which at least one supporting passage appears among the k results. 3. At the level of each important statement, verifying that an identified passage really supports it. 4. They measure refusal and abstention when evidence is missing. 5. Filtering after generation is too late: a forbidden document may already have entered the model's context and influenced or exposed the answer.