Generation Depth Evaluator

You will be given a USER_QUERY and a GENERATED RESPONSE. Your task is to rate the response on DEPTH to evaluate its overall quality. Please approach this evaluation with careful consideration and strive for precision in your assessments.

Evaluation Steps:

1. Thoroughly analyze the given USER_QUERY and the GENERATED RESPONSE.

2. Assess the GENERATED RESPONSE based on the following criterion:

   • DEPTH (1-5): For topics of moderate to high complexity, assess whether the response transcends surface-level explanations to provide specific and in-depth information. Evaluate the extent to which the response offers detailed, nuanced, and expert-level insights that demonstrate a comprehensive understanding of the subject matter.

3. Follow these steps:

   • Analyze the response thoroughly, considering all aspects that contribute to the criterion being evaluated.
   • Determine which pair of adjacent scores (e.g., 2-3, 3-4, or 4-5) best represents the quality of the response for this criterion.
   • Estimate the probability distribution between these two adjacent scores, ensuring they sum to 100%. Strive for precision in this estimation.
   • Provide a detailed, step-by-step analysis of your reasoning process, explicitly explaining how you arrived at your score and probability distribution.

4. Calculate the Weighted_Summed_Score for each criterion.

5. Format your evaluation as shown in the Example Output below.

Example Output:

• DEPTH_Reasoning: [Provide a detailed, step-by-step analysis of your reasoning process, explicitly explaining how you arrived at your score and probability distribution.]

• DEPTH_Formula: (5 * 0.9) + (4 * 0.1) = 4.9

• DEPTH_Weighted_Summed_Score: 4.9

USER_QUERY: {user_query}

GENERATED RESPONSE: {submission}