Evaluate AI Response Relevance with OpenAI and Cosine Similarity
This workflow evaluates the relevance of AI agent responses by comparing generated questions with the original user questions using cosine similarity. It helps ensure AI responses are accurate and contextually appropriate.
The workflow is designed for Q&A agents and leverages OpenAI models to generate questions from AI responses. These generated questions are then compared to the original user questions using cosine similarity to determine relevance. A high similarity score indicates a relevant response, while a low score suggests potential issues such as irrelevant information or hallucinations. This process is based on the RAGAS evaluation metric for answer relevance.
New buyers can create an account from the cart to unlock a controlled $10 first-purchase credit on eligible orders of $25+.
Part of a bundle
AI Agent Builder Pack
This workflow is one of 6 in the bundle — $49.00 for all of them.
That is $8.17 each, vs $14.99 for this one alone.
View bundleSecure checkout powered by Stripe
Support
How to import this workflow into n8n
- 1Purchase or download the workflow to get the n8n workflow JSON file.
- 2In your n8n instance, open Workflows and choose "Import from File" (or paste the JSON with Ctrl+V on the canvas).
- 3Open each node marked with a credential warning and connect your own accounts and API keys.
- 4Run the workflow once manually to verify the data flow, then toggle it to Active.
Related AI workflows
- Launch Your First AI-Powered Chatbot with Actionable Tools$9.99
- Build a WhatsApp Assistant with Memory, Google Suite, Multi-AI, Research, and Imaging$24.99
- Automate AI Video Creation and YouTube Upload with Google Sheets$14.99
- Automate Blog Post Creation and Publishing with GPT, Leonardo AI, and WordPress$14.99
- Automate SEO Keyword Generation with ChatGPT from Google Sheets$3.99
- Email Agent$500.99
More from Eli Brooks
Need this deployed? We'll set it up for you.
Our automation experts deploy this workflow in your stack, connect your accounts, and verify it works — or build a custom solution from scratch.