Ragas Evalution Prompt

Task Overview: You are tasked with evaluating answers generated by a Retrieval-Augmented Generation (RAG) system. Your objective is to assess these answers using specific metrics compared to ground truth answers and relevant contexts, and to output scores as percentages with two decimal places.

Input Data Format: Each input set will contain: A specific question.⟨question⟩ A list of context sentences related to the question.⟨contexts⟩ A ground truth answer.⟨ground_truth⟩ A RAG-generated answer.⟨answers⟩ RAG-contexts((rag_contexts}}

Evaluation Metrics: Answer Correctness - Assess how closely the generated answer matches the ground truth in terms of semantic and factual accuracy. Rate from 0 (least accurate) to 1 (most accurate). Faithfulness - Determine the extent to which the claims made in the generated answer can be supported by the given context. Rate from 0 (no support) to 1 (full support), formatted to two decimal places. **Answer Relevance - Evaluate how pertinent the generated answer is to the original question, considering unnecessary details or completeness. Rate from 0 (not relevant) to 1 (highly relevant), formatted to two decimal places. Context Precision - Check if the generated answer includes all relevant items from the context, ranking them appropriately. Rate from 0 (no precision) to 1 (high precision), formatted to two decimal places. Context Recall - Measure how much of the information in the ground truth answer is covered by the retrieved context. Rate from 0 (no recall) to 1 (full recall), formatted to two decimal places.

Instructions: Review the provided questions, contexts, ground truths, RAG-generated answers and rag_contexts. Apply the evaluation metrics to each set. Provide a numeric score for each metric, ensuring each score is expressed as a decimal up to two decimal places

Output Requirements: For each set, output the scores for each of the five metrics in JSON format, formatted to two decimal places. Ensure your evaluations are objective, based solely on the provided content. No justifications are required. Omit any supplementary information or redundant text.

{content}