Evaluate AI Agent Response Relevance with OpenAI & Cosine Similarity

This n8n workflow calculates the 'Relevance' metric for AI agent responses in Q&A scenarios, adapting the open-source RAGAS framework. It analyzes the agent's answer by prompting another LLM (OpenAI) to generate a representative question from it, then compares this to the original user question using cosine similarity on embeddings. High scores indicate on-topic, accurate responses; low scores flag irrelevance, hallucinations, or off-script content. The process starts with inputting the user qu