Neura Market
Berkeley Function-Calling Leaderboard logo

Berkeley Function-Calling Leaderboard

Free

evaluates LLM's ability to call external functions/tools.

FreeFree tier
Type
Open Source
Company
University of California, Berkeley (Gorilla LLM Team)
Visit Website GitHub (38 stars)

About Berkeley Function-Calling Leaderboard

The Berkeley Function Calling Leaderboard (BFCL) V4, developed by the Gorilla LLM team at UC Berkeley, evaluates large language models' ability to accurately call functions (tools) using real-world data. It introduces multiple evaluation versions (v1 through v4) with progressive features: AST-based accuracy metrics, enterprise and OSS-contributed functions, multi-turn interactions, and holistic agentic evaluation. The leaderboard provides cost estimates, latency measurements, overall accuracy scores, format sensitivity tests, and an interactive error type analysis treemap. Models are evaluated via native function-calling support or prompt-based walk-arounds. The dataset, code, and evaluation methodology are open-source, and the leaderboard is updated periodically.

Key Features

Real-world function calling dataset
Multiple evaluation versions: v1 (AST metric), v2 (enterprise/OSS functions), v3 (multi-turn), v4 (holistic agentic evaluation)
Interactive leaderboard with overall accuracy, cost, and latency metrics
Format sensitivity test cases for prompt-based models
Error type analysis treemap for detailed error breakdown
Open-source code and data with reproducibility via GitHub and PyPI package
Function calling demo with code and OpenAI-compatible output
Supports native function-calling and prompt-based evaluation modes

Pros & Cons

Pros
  • Comprehensive and evolving benchmark with multiple evaluation dimensions
  • Uses real-world, practical function calling scenarios
  • Open-source, reproducible, and community-contributed test cases
  • Detailed error analysis helps identify model weaknesses
  • Regularly updated with new versions and models
Cons
  • Focused solely on function calling, not general LLM evaluation
  • Complex metric composition may be difficult to interpret
  • Limited to the test cases contributed by the community and authors

Best For

Benchmarking LLMs for tool/function calling capabilitiesComparing model performance in real-world function execution tasksResearch in agentic AI and multi-turn tool useEvaluating cost and latency trade-offs across models

FAQ

How can I add my model to the leaderboard?
You can contact the team via Discord (link on the website) to request adding a model or contributing test cases.
What metrics does the leaderboard use?
It uses overall accuracy (unweighted average of sub-categories), cost estimate for the entire benchmark in USD, latency in seconds, and format sensitivity tests. Detailed score composition is explained in the BFCL blog.
Are the evaluation code and data publicly available?
Yes, the code and data are open-source. You can reproduce results by checking out the codebase at the specified commit or installing the PyPI package bfcl-eval.