Neura Market
scrapling-fetch logo

scrapling-fetch

Free

从有反爬虫措施的网站获取文本内容。

FreeFree tier
Inputs: urlOutputs: text
Type
Open Source
Visit Website GitHub (40 stars)

About scrapling-fetch

scrapling-fetch-mcp is an MCP server that enables AI assistants to fetch text content from websites with anti-automation measures. It uses Scrapling (leveraging patchright and curl-cffi) to bypass bot protection and returns clean HTML or Markdown. The tool is optimized for low-volume retrieval of documentation and reference materials, not for high-volume scraping. It provides two main capabilities: page fetching with pagination support and regex-based pattern extraction. Three protection levels are automatically escalated: basic (fast), stealth (moderate, headless Chromium), and max-stealth (thorough, full browser fingerprint). Installation is available as a Claude Code skill or as an MCP server for Claude Desktop. The project is open source under Apache 2.0.

Key Features

Page fetching with pagination support for complete page retrieval
Regex-based pattern extraction to find specific content
Three escalating protection levels: basic, stealth, max-stealth (headless browser)
Returns clean HTML or Markdown output
Automatic bypass of anti-automation measures using Scrapling
Optimized for low-volume documentation and reference material retrieval
Easy installation as a Claude Code skill or MCP server for Claude Desktop

Pros & Cons

Pros
  • Effectively bypasses common anti-automation measures
  • Automatically escalates protection levels based on site difficulty
  • Supports pagination for multi-page content retrieval
  • Returns clean HTML or Markdown, not raw page source
  • Free and open source under Apache 2.0 license
  • Seamless integration with Claude Code and Claude Desktop
Cons
  • Text content only (no images, PDFs, or non-text elements)
  • Not designed for high-volume scraping or data harvesting
  • Initial browser binary download is several hundred MB
  • Performance varies significantly by site complexity and protection level
  • May timeout on first use while browser binaries are downloading
  • Requires Python 3.10+ and the uv package manager

Best For

Fetching documentation from bot-protected websites for AI assistantsRetrieving reference materials and articles with anti-automation measuresExtracting specific content via regex pattern matching from web pagesAutomated reading of API docs and installation instructionsProviding context to AI models from websites that block standard scrapers

FAQ

How does scrapling-fetch-mcp bypass anti-automation measures?
It uses Scrapling, which leverages patchright and curl-cffi to mimic real browser behavior. The tool automatically escalates through three protection levels (basic, stealth, max-stealth) as needed.
Can I install it as a Claude Code skill?
Yes, you can clone the repository and copy the skill folder into your project's .claude/skills/ directory or globally into ~/.claude/skills/. The first invocation runs a one-time setup.
What are the three protection levels and their performance?
Basic is fast (1-2 seconds) and works for most sites. Stealth is moderate (3-8 seconds) using headless Chromium. Max-stealth is thorough (10+ seconds) with full browser fingerprinting.
Is this tool suitable for large-scale scraping?
No, it is explicitly optimized for low-volume retrieval of documentation and reference materials, not for high-volume scraping or data harvesting.