RAG Document Relevance Evaluation Metric for n8n AI Workflows

This n8n workflow template demonstrates the evaluation feature for assessing retrieved document relevance in Retrieval-Augmented Generation (RAG) setups. It processes a question, retrieves documents from a vector store via an agent, and uses AI to score how relevant the retrieved information is to the query. The top section ingests sample documents into the vector store once, while the main flow handles both regular executions and evaluations triggered by n8n's dataset input.

Key mechanics include parallel triggers (standard and evaluation), agent configuration to output tool data (retrieved docs), and conditional AI-based relevance scoring only during evaluations to optimize costs. Metrics are returned to n8n for analysis, highlighting workflow strengths and weaknesses across test cases.

Benefits include building confidence in AI reliability, quick identification of retrieval issues, and cost-efficient testing without full production runs. Ideal for RAG Q&A agents, chatbots, or knowledge bases in e-commerce, where accurate product/payment info retrieval prevents errors.

Use cases: Validate RAG pipelines for customer support (e.g., payment queries), document search in e-commerce catalogs, or compliance checks. Enhances AI workflows in text analytics, saving debugging time and improving accuracy.