Evaluate AI Agent Response Correctness with OpenAI & RAGAS

Evaluates AI agent response accuracy against ground truths using RAGAS methodology, classifying into TP/FP/FN and averaging similarity scores for a final correctness metric.

This n8n workflow implements the 'Correctness' evaluation metric from the RAGAS open-source project to assess AI agent responses. It compares the agent's verbose, conversational output against multiple ground truths, classifying extracted statements into True Positives (present in both), False Positives (in agent response but not ground truth), and False Negatives (in ground truth but missing from response). An average semantic similarity score is also computed using OpenAI embeddings. The fina
Platform
n8n
Category
Finance & Accounting
Price
$19.99
Creator
Leandro Rossi

How to import this workflow into n8n

  1. 1Purchase or download the workflow to get the n8n workflow JSON file.
  2. 2In your n8n instance, open Workflows and choose "Import from File" (or paste the JSON with Ctrl+V on the canvas).
  3. 3Open each node marked with a credential warning and connect your own accounts and API keys.
  4. 4Run the workflow once manually to verify the data flow, then toggle it to Active.

Related Finance & Accounting workflows

More from Leandro Rossi

All Finance & Accounting workflows →All n8n workflows →

Need this deployed? We'll set it up for you.

Our automation experts deploy this workflow in your stack, connect your accounts, and verify it works — or build a custom solution from scratch.

Get It DeployedCustom Build