Project that scape all github repositories, read them, analyse with an AI and return you on a discord webhook project that could interest you.

GitHub Scraping AI

AI-powered GitHub project discovery. Automatically finds interesting repositories matching your interests and sends daily notifications to Discord.

Features

• Smart Filtering: Uses LLM to evaluate repositories against your natural language interests
• Keyword Search: Filter GitHub search with custom keywords (OR logic)
• Fork Exclusion: Automatically filters out forked repositories
• Multi-Provider Support: Works with OpenAI, Anthropic (Gemini), or Google (Gemini)
• Async & Concurrent: Fetches READMEs concurrently for faster processing
• Daily Discovery: Fetches top starred repos from the last 24 hours
• Discord Notifications: Rich embeds with repo details and AI reasoning
• Rejected Repos Log: Logs rejected repos with reasons for prompt fine-tuning
• Deduplication: Tracks seen repos to avoid duplicates
• Dry Run Mode: Test without sending to Discord

How It Works

GitHub API → Fetch top 1000 new repos → LLM evaluates each against your prompt → Discord notification

1. Queries GitHub for repositories created in the last 24h, sorted by stars
2. Fetches README excerpts for context
3. Sends each repo + your interests to an LLM for evaluation
4. Posts matching repos to Discord with the AI's reasoning
5. Caches seen repos to prevent duplicates

Quick Start

Prerequisites

• Python 3.13+
• uv package manager
• API key for one of: OpenAI, Anthropic, or Google
• GitHub personal access token
• Discord webhook URL

Installation

# Clone the repository
git clone https://github.com/yourusername/github-scraping-agent-ai.git
cd github-scraping-agent-ai

# Install dependencies
uv sync

Configuration

1. Copy example files:

   cp config.example.json config.json
   cp prompt.example.md prompt.md
   

2. Edit config.json with your credentials:

   {
     "github": {
       "token": "ghp_your_github_token",
       "key