AI Answer Similarity Evaluation Metric

This n8n workflow implements the 'Answer Similarity' evaluation metric from the RAGAS framework, computing cosine similarity between embeddings of an AI-generated response and the expected ground truth answer. It excels in scenarios with factual, close-ended questions where minimal deviation is anticipated, providing a quantitative score from 0 to 1—higher scores indicate strong alignment and consistency.

Key benefits include automated quality assessment of LLM outputs, early detection of hallucinations or inconsistencies, and seamless integration into evaluation pipelines. By leveraging OpenAI embeddings, it saves manual review time and scales for large datasets, making it ideal for refining AI agents in production.

Use cases span chatbot validation, RAG system testing, customer support response auditing, and compliance checks in commerce environments. Pair it with sample data from the provided Google Sheet to benchmark models like GPT-4o-mini, ensuring reliable performance with n8n version 1.94+.