bigcodebench

Name: bigcodebench
Author: bigcode-project

bigcode-project April 29, 2024

495 copies 0 downloads

[ICLR'25] BigCodeBench: Benchmarking Code Generation Towards AGI

Agent Definition

# BigCodeBench
<center>
<img src="https://github.com/bigcode-bench/bigcode-bench.github.io/blob/main/asset/bigcodebench_banner.svg?raw=true" alt="BigCodeBench">
</center>

<p align="center">
    <a href="https://huggingface.co/spaces/bigcode/bigcodebench-leaderboard"><img src="https://img.shields.io/badge/🤗&nbsp&nbsp%F0%9F%8F%86-leaderboard-%23ff8811"></a>
    <a href="https://huggingface.co/collections/bigcode/bigcodebench-666ed21a5039c618e608ab06"><img src="https://img.shields.io/badge/🤗-collection-pink"></a>
    <a href="https://bigcode-bench.github.io/"><img src="https://img.shields.io/badge/%F0%9F%8F%86-website-8A2BE2"></a>
    <a href="https://arxiv.org/abs/2406.15877"><img src="https://img.shields.io/badge/arXiv-2406.15877-b31b1b.svg"></a>
    <a href="https://pypi.org/project/bigcodebench/"><img src="https://img.shields.io/pypi/v/bigcodebench?color=g"></a>
    <a href="https://pepy.tech/project/bigcodebench"><img src="https://static.pepy.tech/badge/bigcodebench"></a>
    <a href="https://github.com/bigcodebench/bigcodebench/blob/master/LICENSE"><img src="https://img.shields.io/pypi/l/bigcodebench"></a>
    <a href="https://hub.docker.com/r/bigcodebench/bigcodebench-evaluate" title="Docker-Eval"><img src="https://img.shields.io/docker/image-size/bigcodebench/bigcodebench-evaluate"></a>
</p>

<p align="center">
    <a href="#-impact">💥 Impact</a> •
    <a href="#-news">📰 News</a> •
    <a href="#-quick-start">🔥 Quick Start</a> •
    <a href="#-remote-evaluation">🚀 Remote Evaluation</a> •
    <a href="#-llm-generated-code">💻 LLM-generated Code</a> •
    <a href="#-advanced-usage">🧑 Advanced Usage</a> •
    <a href="#-result-submission">📰 Result Submission</a> •
    <a href="#-citation">📜 Citation</a>
</p>

<div align="center">
    <h2>🎉 Check out our latest work!<br>
    <a href="https://arxiv.org/abs/2510.08697">🌟 BigCodeArena 🌟</a><br>
    <strong>🚀 Open Evaluation Platform on AI for Vibe Coding 🚀<br>
    ✨ 100% free to use the latest frontier

Comments

More Agents

View all

hybrid-model-workflow

HAL 分层混合模型工作流 — 强模型(Claude)负责理解/拆解/验收，低成本模型(DeepSeek)负责检索/提取/清洗。Hermes Agent skill。

ph4ble

Dynamic-Review-Agent

An LLM agent fine-tuned on DeepSeek for spaced repetition, dynamically integrating knowledge points based on the Ebbinghaus forgetting curve.

1838177

StellarOS-Watch

基于 STM32F103 构建的端到端 AI 智能手表生态。自研“零重定位”原生机器码动态加载引擎与页面栈式 UI 框架；集成生产级 OTA 回滚保护机制与高带宽（921600 baud）串口协议栈。通过 Node.js 中继实现 DeepSeek AI 语义控制及 ASRPRO 语音全双工交互，是一个集成了分布式计算、现代存储管理与 AI Agent 的嵌入式全栈工程。

chenshuang888