Neura Market
I submitted 316 AI-generated PRs to open source logo

I submitted 316 AI-generated PRs to open source

Free
FreeFree tier
Type
Open Source
Visit Website GitHub (2 stars)

About I submitted 316 AI-generated PRs to open source

This project details an experiment conducted by June Kim, who submitted 316 AI-generated pull requests to various open source repositories to study maintainer reactions and the protocol dynamics between AI contributors and human maintainers. The pipeline automatically identified repos with open issues, generated fixes using AI, and submitted PRs with a pacing mechanism (the 'drip queue') to avoid overwhelming maintainers. Key findings include that maintainers often reject AI-generated PRs based on provenance rather than code quality, leading to real bugs being left unfixed, maintainer burnout, and account bans. The experiment highlights a protocol problem: as AI improves, the burden shifts from contributors to maintainers, and existing filtering mechanisms are insufficient.

Key Features

Automated PR generation from open issues using AI
Drip queue pacing (one PR per org per merge cycle) to avoid org-wide rejection
Public rejection analysis with maintainer comments (e.g., tinygrad)
Real bug fixes that pass CI and tests
Tracking of maintainer outcomes including account bans
Insights on maintainer burnout and protocol problems

Pros & Cons

Pros
  • Reveals real issues in the interaction between AI contributors and human maintainers
  • Provides data-driven insights on how maintainers respond to AI PRs
  • Fixes real bugs in open source projects
  • Identifies protocol problems rather than blaming people
  • PRs pass CI and automated tests
Cons
  • PRs often rejected based on AI provenance rather than code quality
  • Wastes maintainer time and attention, contributing to burnout
  • Results in account bans from repositories
  • Some PRs may be low quality or not follow community norms
  • Maintainers must invest extra effort to reject and filter submissions

Best For

Research on AI contributions in open source ecosystemsUnderstanding maintainer filtering mechanisms and responses to AI-generated codeTesting the acceptance rate and quality perception of AI-generated patchesIdentifying protocol improvements for AI-human collaboration in open source

FAQ

How many PRs were submitted in the experiment?
316 AI-generated pull requests were submitted to various open source repositories.
What is the drip queue?
The drip queue is a pacing mechanism that limits PR submissions to one per organization per merge cycle to avoid overwhelming maintainers and prevent org-wide rejection.
What did the author learn from the experiment?
The author concluded that open source survives by filtering low-quality submissions, and AI shifts the burden from contributor to maintainer. The defense must be cheap, or maintainers lose by attrition. It is a protocol problem, not a people problem.
What was the acceptance rate for the PRs?
The experiment showed that many PRs were rejected. For example, out of 13 PRs to the tinygrad project, only 1 was merged, and maintainer comments escalated from warnings to bans.