Recursive Web Scraper for LLMs with Airtop

Automates recursive web scraping from Google Sheets using Airtop: scrapes pages, stores content in Docs, discovers and adds new matching links, repeating up to specified depth for comprehensive data collection.

n8n
Recursive Web Scraper for LLMs with Airtop

This n8n workflow enables recursive web scraping ideal for aggregating content across linked pages, perfect for LLM training data, research, lead generation, or content monetization. It starts with seed URLs from a Google Sheet, uses Airtop to scrape each page's content, stores it in a Google Doc, extracts new links matching a specified string (e.g., domain), and appends them back to the sheet. This process iterates based on the 'Depth' parameter (e.g., 3 layers), ensuring broad coverage without manual intervention.

Key benefits include time-saving automation of deep web exploration, scalable content harvesting for LLMs, and easy integration with Google Workspace. Setup requires Airtop API key (free), Google Sheets/Docs credentials. It handles dynamic sites efficiently via Airtop's AI-powered scraping, reducing failures on JavaScript-heavy pages.

Use cases: Build datasets for AI models, monitor competitor sites, collect media/articles for newsletters, or generate leads from link networks. Enhancements like scheduling or filtering boost its power for ongoing projects.

$22.99
Last updated November 14, 2026
30-day money-back guarantee
Instant download
Lifetime updates included

New buyers can create an account from the cart to unlock a controlled $10 first-purchase credit on eligible orders of $25+.

Secure checkout powered by Stripe

Support

How to import this workflow into n8n

  1. 1Purchase or download the workflow to get the n8n workflow JSON file.
  2. 2In your n8n instance, open Workflows and choose "Import from File" (or paste the JSON with Ctrl+V on the canvas).
  3. 3Open each node marked with a credential warning and connect your own accounts and API keys.
  4. 4Run the workflow once manually to verify the data flow, then toggle it to Active.

Related Media & Entertainment workflows

More from BestWorkflows

Need this deployed? We'll set it up for you.

Our automation experts deploy this workflow in your stack, connect your accounts, and verify it works — or build a custom solution from scratch.