Evaluate RAG Response Accuracy with OpenAI: Document Groundedness Metric
### This n8n template demonstrates how to calculate the evaluation metric RAG document groundedness, which in this scenario, measures the ability to provide or reference information included only in retrieved vector store documents.
The scoring approach is adapted from [https://cloud.google.com/vertex-ai/generative-ai/docs/models/metrics-templates#pointwise_groundedness](https://cloud.google.com/vertex-ai/generative-ai/docs/models/metrics-templates#pointwise_groundedness)
### How it works
- This evaluation works best for an agent that requires document retrieval from a vector store or similar source.
- For our scoring, we need to collect the agent's response and the documents retrieved and use an LLM to assess if the former is based off the latter.
- A key factor is to look out for information in the response which is not mentioned in the documents.
- A high score indicates LLM adherence and alignment, whereas a low score could signal inadequate prompt or model hallucination.
### Requirements
- n8n version 1.94+
- Check out this Google Sheet for a sample data [https://docs.google.com/spreadsheets/d/1YOnu2JJjlxd787AuYcg-wKbkjyjyZFgASYVV0jsij5Y/edit?usp=sharing](https://docs.google.com/spreadsheets/d/1YOnu2JJjlxd787AuYcg-wKbkjyjyZFgASYVV0jsij5Y/edit?usp=sharing)
Platform
n8n
Category
AI
Price
Free
Creator
Jimleuk
pointwise_groundedness
set
noOp
evaluation
stickyNote
httpRequest
manualTrigger
agent
evaluationTrigger
chainLlm
How to import this workflow into n8n
1Purchase or download the workflow to get the n8n workflow JSON file.
2In your n8n instance, open Workflows and choose "Import from File" (or paste the JSON with Ctrl+V on the canvas).
3Open each node marked with a credential warning and connect your own accounts and API keys.
4Run the workflow once manually to verify the data flow, then toggle it to Active.