crawl4ai-mcp-server

# Crawl4AI MCP Server

🕷️ **A lightweight Model Context Protocol (MCP) server that exposes [Crawl4AI](https://docs.crawl4ai.com/) web scraping and crawling capabilities as tools for AI agents.**

Similar to Firecrawl's API but self-hosted and free. Perfect for integrating web scraping into your AI workflows with OpenAI Agents SDK, Cursor, Claude Code, and other MCP-compatible tools.

## Features

- **🔧 MCP Tools**: Exposes 4 powerful tools: `scrape`, `crawl`, `crawl_site`, `crawl_sitemap` via stdio MCP server
- **🌐 Web Scraping**: Single-page scraping with markdown extraction
- **🕷️ Web Crawling**: Multi-page breadth-first crawling with depth control
- **🧠 Adaptive Crawling**: Smart crawling that stops when sufficient content is gathered
- **🛡️ Safety**: Blocks internal networks, localhost, and private IPs
- **📱 Agent Ready**: Works with OpenAI Agents SDK, Cursor, and Claude Code
- **⚡ Fast**: Powered by Playwright and Crawl4AI's optimized extraction

## 🚀 Quick Start

Choose between **Docker** (recommended) or **manual installation**:

### Option A: Docker (Recommended) 🐳

Docker eliminates all setup complexity and provides a consistent environment:

#### Option A1: Use Pre-built Image (Fastest) ⚡
```bash
# No setup required! Just pull and run the published image
docker pull uysalsadi/crawl4ai-mcp-server:latest

# Test it works
python test-config.py

# Use directly in MCP configurations (see examples below)
```

#### Option A2: Build Yourself
```bash
# Clone the repository
git clone https://github.com/uysalsadi/crawl4ai-mcp-server.git
cd crawl4ai-mcp-server

# Quick build and test (simplified)
docker build -f Dockerfile.simple -t crawl4ai-mcp .
echo '{"jsonrpc": "2.0", "id": 1, "method": "initialize", "params": {"protocolVersion": "2024-11-05", "capabilities": {}, "clientInfo": {"name": "test", "version": "1.0"}}}' | docker run --rm -i crawl4ai-mcp

# Or use helper script (full build with Playwright)
./docker-run.sh build
./docker-run.sh test
./docker-run.