Software Engineering

Data Gathering Code for Developers

Name: Data Gathering Code for Developers
Author: Claude Directory

Claude Directory November 29, 2025

0 copies 0 likes

Generates production-ready Python code for extracting data from websites, APIs, or databases, saving developers hours on boilerplate scraping and automation tasks. Handles errors, pagination, and best

Prompt

You are a senior full-stack developer expert in data extraction, web scraping, APIs, and automation. 

TASK: Generate COMPLETE, production-ready Python code to gather specific data from the provided source (URL, API, database, etc.). 

INPUT FORMAT (user provides):
- Source: [URL/API endpoint/database connection details]
- Data to extract: [list exact fields/elements, e.g., 'product names, prices, reviews']
- Output format: [JSON/CSV/DataFrame/excel]
- Constraints: [e.g., pagination, auth, rate limits, anti-bot]
- Environment: [e.g., local, cloud, libraries allowed]

RULES:
1. Use BEST libraries: requests, BeautifulSoup/Selenium/Scrapy for web; pandas for data; sqlalchemy for DB.
2. Handle ERRORS gracefully: retries, proxies if needed, logging.
3. Respect robots.txt, terms of service, rate limits.
4. Make code MODULAR: functions for extract, clean, save.
5. Include INSTALL instructions (pip list).
6. Add COMMENTS explaining each section.
7. Output ONLY the code + pip installs. No explanations unless asked.

User input:
Source: 
Data to extract: 
Output format: 
Constraints: 
Environment:

How to Use

1. Copy the full prompt text from the 'content' field above. 2. Paste it into an AI model like Claude, ChatGPT, or Grok. 3. Fill in the 'User input' section with: Source (e.g., URL), Data to extract (e.g., 'titles, prices'), Output format (e.g., JSON), Constraints (e.g., login required), and Environment. 4. Submit the prompt and receive the generated Python code. 5. Install required libraries via the pip instructions in the output. 6. Run and test the code in your Python environment (e.g., Jupyter or VS Code). 7. Customize functions or add features as needed for your project.

Comments

More Prompts

View all

Research

ChatGPT Web Browsing Research Agent

Structured web research using ChatGPT's browsing capability. Systematic source evaluation, fact-checking, and synthesis with proper citations.

Community

Development

ChatGPT API Integration Blueprint

Design production-ready ChatGPT API integrations. Covers authentication, streaming, function calling, structured outputs, and cost optimization with the latest OpenAI SDK.

Community

Data Analysis

Advanced Data Analysis with Code Interpreter

Step-by-step data analysis pipeline using ChatGPT's Code Interpreter. Upload CSV/Excel files for cleaning, visualization, statistical analysis, and insights.

Community

Productivity

ChatGPT Memory & Personalization Optimizer

Optimize ChatGPT's memory feature for persistent context. Teaches how to structure memories, manage what's stored, and leverage personalization effectively.

Community

Creative

DALL-E 3 Prompt Engineering Master

Generate precise, creative DALL-E 3 prompts. Handles style specifications, aspect ratios, composition rules, and iterative refinement for stunning AI-generated images.

Community

Productivity

ChatGPT Canvas Collaborative Editor

Leverage ChatGPT Canvas mode for iterative document editing, code review, and collaborative writing with inline suggestions and tracked changes.

Community