Summarization Evaluation Metric for LLM Outputs

This n8n workflow implements the 'Summarization' evaluation metric to measure an LLM's performance in producing faithful summaries from incoming YouTube transcripts. Adapted from Google Vertex AI's pointwise summarization quality metrics, it directly compares the generated summary to the original transcript, flagging any extraneous information or hallucinations that deviate from the source material. The workflow operates by first checking if evaluation is required via the 'Is Evaluating?' node,