SearXNG Search Reference

SearXNG is a privacy-respecting meta-search tool that queries over 70 search engines simultaneously through a single API. It can be used via a public instance or self-hosted for full control. No API key is required when using a public instance.

When to Use SearXNG

• When you need privacy-respecting web search without tracking.
• When the main web search toolset (FIRECRAWL_API_KEY) is not configured, SearXNG automatically becomes the fallback.
• When you want to aggregate results from multiple search engines in one query.

Prerequisites

Before using SearXNG, ensure the following are in place:

• The SEARXNG_URL environment variable must be set to a valid SearXNG instance URL. Example values:

# Public instances (no setup required)
SEARXNG_URL=https://searxng.example.com

# Self-hosted SearXNG
SEARXNG_URL=http://localhost:8888

• For CLI method: curl must be installed.
• For Python method: requests library must be installed.
• For self-hosting: Docker or pip installed, and appropriate permissions to run containers or services.

Checking SearXNG Availability

Before using SearXNG, verify the instance is reachable:

1. Ensure SEARXNG_URL environment variable is set.
2. Test reachability with a quick request:

# Check if SEARXNG_URL is set and the instance is reachable
curl -s --max-time 5 "${SEARXNG_URL}/search?q=test&format=json" | head -c 200

3. If SEARXNG_URL is set and the instance responds, use SearXNG.
4. If SEARXNG_URL is unset or unreachable, fall back to other available search tools.
5. If the user wants SearXNG specifically, help them set up an instance or find a public one.

CLI Search via curl (Preferred Method)

The recommended way to query SearXNG is through curl in a terminal. Construct the URL with the query and parameters, then parse the JSON response.

Parameters:

Parameter	Required	Description
q	Yes	Query string (URL-encoded)
format	Recommended	Output format: json, csv, rss. Always request format=json explicitly.
engines	No	Comma-separated list of search engines (e.g., engines=google,bing,ddg)
limit	No	Maximum results per engine (e.g., limit=5)
categories	No	Filter by category: general, news, science, etc. (e.g., categories=news,science)
safesearch	No	Safesearch level: safesearch=0 (none), 1 (moderate), 2 (strict)
time_range	No	Filter by time: day, week, month, year (e.g., time_range=week)

Examples:

# Text search (JSON output)
curl -s --max-time 10 \
  "${SEARXNG_URL}/search?q=python+async+programming&format=json&engines=google,bing&limit=10"

# With Safesearch off
curl -s --max-time 10 \
  "${SEARXNG_URL}/search?q=example&format=json&safesearch=0"

# Specific categories (general, news, science, etc.)
curl -s --max-time 10 \
  "${SEARXNG_URL}/search?q=AI+news&format=json&categories=news"

Parsing JSON Results:

The JSON response contains a results array. Each result object includes these fields: title, url, content (snippet), engine, parsed_url, img_src, thumbnail, author, published_date. Example extraction:

# Extract titles and URLs from JSON
curl -s --max-time 10 "${SEARXNG_URL}/search?q=fastapi&format=json&limit=5" \
  | python3 -c "
import json, sys
data = json.load(sys.stdin)
for r in data.get('results', []):
    print(r.get('title',''))
    print(r.get('url',''))
    print(r.get('content','')[:200])
    print()
"

Python API Search via requests

For programmatic access, use the requests library. Always URL-encode queries using urllib.parse.quote() or pass them as parameters.

import os, requests, urllib.parse

base_url = os.environ.get("SEARXNG_URL", "")
if not base_url:
    raise RuntimeError("SEARXNG_URL is not set")

query = "fastapi deployment guide"
params = {
    "q": query,
    "format": "json",
    "limit": 5,
    "engines": "google,bing",
}

resp = requests.get(f"{base_url}/search", params=params, timeout=10)
resp.raise_for_status()
data = resp.json()

for r in data.get("results", []):
    print(r["title"])
    print(r["url"])
    print(r.get("content", "")[:200])
    print()

Self-Hosting SearXNG

Self-hosting provides reliability and avoids rate limits of public instances.

Option A: Docker

# Using Docker
docker run -d -p 8888:8080 \
  -v $(pwd)/searxng:/etc/searxng \
  searxng/searxng:latest

# Then set
SEARXNG_URL=http://localhost:8888

Option B: pip

pip install searxng
# Edit /etc/searxng/settings.yml
searxng-run

Search Then Extract Full Page Content

SearXNG returns only snippets (titles, URLs, short content), not full page content. To get full articles, use web_extract, browser tools, or curl.

# Search for relevant pages
curl -s "${SEARXNG_URL}/search?q=fastapi+deployment&format=json&limit=3"
# Output: list of results with titles and URLs

# Then extract the best URL with web_extract

Constraints and Caveats

• SearXNG returns only snippets. Use web_extract, browser tools, or curl for full articles.
• Instance availability is critical: if the SearXNG instance is down or unreachable, search fails.
• Public instances may have rate limits; self-hosting avoids this.
• Available engines depend on the SearXNG instance configuration; some engines may be disabled.
• Results freshness depends on the external engines SearXNG aggregates.
• Always set SEARXNG_URL; without it the skill cannot function.
• Always URL-encode queries (spaces and special characters) in curl, or use urllib.parse.quote() in Python.
• Always request format=json explicitly; default format may not be machine-readable.
• Always set a timeout (--max-time or timeout=) to avoid hanging on unreachable instances.
• Self-hosting is recommended for reliability over public instances.

Failure Modes

• SEARXNG_URL not set: skill unavailable, agent falls back to other search options.
• Connection refused: instance not running or wrong URL.
• Empty results: instance blocks the query; try a different instance or self-host.
• Slow responses: public instance under load; self-host or use a less-loaded instance.
• json format not supported: old SearXNG version; try format=rss or upgrade SearXNG.
• Rate limiting by public instance.