### This n8n template demonstrates how to calculate the evaluation metric Similarity which in this scenario, measures the consistency of the agent.
The scoring approach is adapted from the open-source evaluations project [RAGAS](https://docs.ragas.io/) and you can see the source here [https://github.com/explodinggradients/ragas/blob/main/ragas/src/ragas/metrics/_answer_similarity.py](https://github.com/explodinggradients/ragas/blob/main/ragas/src/ragas/metrics/_answer_similarity.py)
### How it works
- This evaluation works best where questions are close-ended or about facts where the answer can have little to no deviation.
- For our scoring, we generate embeddings for both the AI's response and ground truth and calculate the cosine similarity between them.
- A high score indicates LLM consistency with expected results whereas a low score could signal model hallucination.
### Requirements
- n8n version 1.94+
- Check out this Google Sheet for a sample data [https://docs.google.com/spreadsheets/d/1YOnu2JJjlxd787AuYcg-wKbkjyjyZFgASYVV0jsij5Y/edit?usp=sharing](https://docs.google.com/spreadsheets/d/1YOnu2JJjlxd787AuYcg-wKbkjyjyZFgASYVV0jsij5Y/edit?usp=sharing)
Platform
n8n
Category
AI & Machine Learning
Price
Free
Creator
Jimleuk
set
code
noOp
merge
splitOut
aggregate
evaluation
stickyNote
httpRequest
agent
How to import this workflow into n8n
1Purchase or download the workflow to get the n8n workflow JSON file.
2In your n8n instance, open Workflows and choose "Import from File" (or paste the JSON with Ctrl+V on the canvas).
3Open each node marked with a credential warning and connect your own accounts and API keys.
4Run the workflow once manually to verify the data flow, then toggle it to Active.