Recursive Web Scraper for LLMs with Airtop - n8n Workflow | Neura Market
Recursive Web Scraper for LLMs with Airtop
Automates recursive web scraping from Google Sheets using Airtop: scrapes pages, stores content in Docs, discovers and adds new matching links, repeating up to specified depth for comprehensive data collection.
This n8n workflow enables recursive web scraping ideal for aggregating content across linked pages, perfect for LLM training data, research, lead generation, or content monetization. It starts with seed URLs from a Google Sheet, uses Airtop to scrape each page's content, stores it in a Google Doc, extracts new links matching a specified string (e.g., domain), and appends them back to the sheet. This process iterates based on the 'Depth' parameter (e.g., 3 layers), ensuring broad coverage without
Platform
n8n
Category
Media & Entertainment
Price
$22.99
Creator
BestWorkflows
web scraping
recursive scraping
Airtop
Google Sheets
Google Docs
LLMs
content extraction
data collection
automation
AI data pipeline
How to import this workflow into n8n
1Purchase or download the workflow to get the n8n workflow JSON file.
2In your n8n instance, open Workflows and choose "Import from File" (or paste the JSON with Ctrl+V on the canvas).
3Open each node marked with a credential warning and connect your own accounts and API keys.
4Run the workflow once manually to verify the data flow, then toggle it to Active.