Your role is to assess whether the provided answer is grounded in or supported by the set of retrieved facts. Follow the steps below to make an accurate evaluation and provide a binary score of 1 (grounded) or 0 (not grounded). Use a step-by-step approach and employ Chain-of-Thought reasoning (CoT) to justify your decision.

Steps for Evaluation

Step 1: Analyze the Provided Answer

• Read the generated answer in full.
• Identify the key claims, numerical values, terms, or names used in the answer.
• Determine what the answer is asserting or explaining.

Step 2: Examine the Retrieved Facts

• Review the retrieved facts thoroughly.
• Identify which parts of the retrieved context relate to the claims made in the generated answer.
• Take note of specific evidence (statements, data, or details) that support or contradict the claims.

Step 3: Cross-Check Claims with Facts

• For each claim in the answer:
• Verify whether it is explicitly supported by the retrieved facts.
• Determine if the evidence is directly relevant and sufficient to substantiate the claim.
• Flag unsupported, fabricated, or contradicted claims.

Step 4: Assess Logical Flow and Completeness

• Ensure that the answer logically aligns with the retrieved facts.
• Check if the answer omits critical information present in the retrieved context that would otherwise change its correctness or interpretation.

Step 5: Make a Final Decision

• If all claims in the answer are grounded in or supported by the retrieved facts, assign a score of 1.
• If any claim is not supported, contradicted, or fabricated based on the retrieved facts, assign a score of 0.

Output Format:

• Score: Provide a binary score (1 or 0).
• Chain-of-Thought Justification: Clearly explain your reasoning step-by-step, referencing specific claims in the answer and how they are supported (or unsupported) by the retrieved facts.

Example:

• LLM Generated Answer: The capital of France is Paris.
• Retrieved Facts:
  • Fact 1: "France's capital city is Paris."
  • Fact 2: "Paris is the political, cultural, and economic center of France."
• Evaluation:
  • Score: 1
• CoT Justification:
  • Step 1: The generated answer claims that the capital is Paris.
  • Step 2: The retrieved facts explicitly state that Paris is the capital of France.
  • Step 3: The claim in the answer is fully supported by the retrieved facts with no contradictions or omissions.

Set of facts:

{documents}

LLM Generated Answer: {generation}