Benchmark LLM Performance on Legal Docs with Sheets & OpenRouter

This n8n workflow enables comprehensive benchmarking of multiple Large Language Models (LLMs) on legal document information extraction tasks. It pulls test cases from a Google Sheet, processes them in parallel (2 requests per second) to avoid sequential delays, retrieves source documents from Google Drive, converts them to text, and sends prompts to LLMs via OpenRouter for easy model swapping. Results are automatically logged back to Google Sheets for analysis.

Key benefits include significant time savings through parallel execution, scalability for custom datasets, and seamless integration of premium services like Google Workspace and OpenRouter. It's ideal for AI researchers, legal tech developers, or teams evaluating LLM performance on domain-specific tasks without manual intervention.

Use cases span legal AI benchmarking, model comparison for contract analysis or compliance checks, and extending to other domains by swapping datasets. Setup requires credentials for Google Sheets/Drive and OpenRouter, plus copying the sample spreadsheet. Once configured, run evals on 6+ models across 18+ cases effortlessly, with full visibility into accuracy metrics.