AI Correctness Evaluation Metric Template for n8n

This n8n workflow serves as a template for implementing the evaluation feature, specifically calculating a 'correctness' metric. It processes a test dataset of inputs (e.g., questions about historical events) through an AI workflow and uses another AI model to judge if the generated output matches the meaning of the expected reference answer. The workflow supports dual triggering: a regular chat trigger for normal use and an evaluation trigger for testing, ensuring metrics are only computed during evaluations to optimize costs.

Key components include parallel triggers, conditional logic to detect evaluation mode, and an AI node that assesses semantic similarity between actual and expected outputs. The metric is then passed back to n8n's evaluation system, providing scores for each test case to identify workflow strengths and weaknesses.

Benefits include building confidence in AI reliability, iterative improvements based on quantitative scores, and cost efficiency by skipping evaluations in production. Ideal for developers refining LLM-based automations.

Use cases span AI chatbots, Q&A systems, text generation workflows, and any n8n setup needing output validation, such as customer support bots in hospitality or analytics tools.