Web Search, X Search, and Web Fetch Tools

Learn how to use web_search, x_search, and web_fetch for web searches, X content retrieval, and URL fetching. This guide covers provider selection, configuration, and usage for developers.

Read this when

  • You want to enable or configure web_search
  • You want to enable or configure x_search
  • You need to choose a search provider
  • You want to understand auto-detection and provider selection

web_search performs web searches through your chosen provider and delivers consistent results, with query caching lasting 15 minutes (adjustable). OpenClaw additionally includes x_search for X (previously Twitter) content and web_fetch for simple URL retrieval. web_fetch operates entirely on your machine; web_search goes through xAI Responses when Grok serves as the provider, and x_search consistently uses xAI Responses.

Info

web_search is a basic HTTP tool, not a browser. For sites that rely heavily on JavaScript or require login credentials, turn to the Web Browser. To retrieve a particular URL, use Web Fetch.

Quick start

Choose a provider

Select a provider and finish any necessary configuration. Certain providers require no key, while others demand an API key. Consult the individual provider pages for specifics.

Configure

openclaw configure --section web

This command saves the provider and any required credentials. For providers backed by an API, you can alternatively set the provider's environment variable (such as BRAVE_API_KEY) and omit this step.

Use it

await web_search({ query: "OpenClaw plugin SDK" });

For X posts:

await x_search({ query: "dinner recipes" });

Choosing a provider

  • Brave Search, Organized results that include excerpts. Offers llm-context mode along with country and language filters. A free tier exists.

  • Codex Hosted Search, AI-generated grounded responses using your Codex app-server account.

  • DuckDuckGo, No key required. No API key necessary. Uses an unofficial HTML-based integration.

  • Exa, Combines neural and keyword search with content extraction features such as highlights, text, and summaries.

  • Firecrawl, Delivers structured results. Works optimally when combined with firecrawl_search and firecrawl_scrape for deeper extraction.

  • Gemini, AI-generated answers with citations, powered by Google Search grounding.

  • Grok, AI-generated answers with citations, powered by xAI web grounding.

  • Kimi, AI-generated answers with citations, powered by Moonshot web search; ungrounded chat fallbacks fail with an error.

  • MiniMax Search, Structured results retrieved through the MiniMax Token Plan search API.

  • Ollama Web Search, Performs searches via a signed-in local Ollama instance or the hosted Ollama API.

  • Parallel, A paid Parallel Search API (PARALLEL_API_KEY) offering higher rate limits and objective tuning.

  • Parallel Search (Free), A key-free opt-in option. Uses Parallel's free Search MCP, providing LLM-optimized dense excerpts without requiring an API key.

  • Perplexity, Structured results with controls for content extraction and domain filtering.

  • SearXNG, A self-hosted meta-search engine. No API key needed. Gathers results from Google, Bing, DuckDuckGo, and others.

  • Tavily, Structured results featuring search depth, topic filtering, and tavily_extract for URL extraction.

Provider comparison

ProviderResult styleFiltersAPI key
BraveStructured snippetsCountry, language, time, llm-context modeBRAVE_API_KEY
Codex Hosted SearchAI-synthesized + source URLsDomains, context size, user locationNone; uses Codex/OpenAI sign-in
DuckDuckGoStructured snippets--None (key-free)
ExaStructured + extractedNeural/keyword mode, date, content extractionEXA_API_KEY
FirecrawlStructured snippetsVia firecrawl_search toolFIRECRAWL_API_KEY
GeminiAI-synthesized + citations--GEMINI_API_KEY
GrokAI-synthesized + citations--xAI OAuth, XAI_API_KEY, or plugins.entries.xai.config.webSearch.apiKey
KimiAI-synthesized + citations; fails on ungrounded chat fallbacks--KIMI_API_KEY / MOONSHOT_API_KEY
MiniMax SearchStructured snippetsRegion (global / cn)MINIMAX_CODE_PLAN_KEY / MINIMAX_CODING_API_KEY / MINIMAX_OAUTH_TOKEN
Ollama Web SearchStructured snippets--None for signed-in local hosts; OLLAMA_API_KEY for direct https://ollama.com search
ParallelDense excerpts ranked for LLM context--PARALLEL_API_KEY (paid)
Parallel Search (Free)Dense excerpts ranked for LLM context--None (free Search MCP)
PerplexityStructured snippetsCountry, language, time, domains, content limitsPERPLEXITY_API_KEY / OPENROUTER_API_KEY
SearXNGStructured snippetsCategories, languageNone (self-hosted)
TavilyStructured snippetsVia tavily_search toolTAVILY_API_KEY

Result shape

At the core tool boundary, web_search standardizes every bundled and external plugin provider. Callers always receive one of these closed shapes:

type WebSearchOutput =
  | {
      kind: "error";
      provider: string;
      error: "provider_error";
      message: string;
      docs?: string;
    }
  | {
      kind: "results";
      provider: string;
      query: string;
      count: number;
      tookMs?: number;
      results: Array<{
        title: string;
        url: string;
        snippet?: string;
        published?: string;
        siteName?: string;
      }>;
      externalContent: {
        untrusted: true;
        source: "web_search";
        wrapped: true;
        provider: string;
      };
      cached?: true;
    }
  | {
      kind: "answer";
      provider: string;
      query: string;
      tookMs?: number;
      content: string;
      citations?: Array<{ url: string; title?: string }>;
      externalContent: {
        untrusted: true;
        source: "web_search";
        wrapped: true;
        provider: string;
      };
      cached?: true;
    }
  | {
      kind: "raw";
      provider: string;
      data: unknown;
    };

Structured providers employ kind: "results", while synthesized providers use kind: "answer". For compatibility, external plugin providers whose payloads do not match either shape pass through unchanged as kind: "raw". Provider-specific details like raw scores, excerpts, related searches, inline-citation offsets, model ids, or session metadata are not forwarded on normalized branches. When a provider's richer response is part of your workflow, use its dedicated tool.

externalContent.wrapped: true acts as a trust marker that the boundary itself guarantees to be true. Provider prose (title, snippet, siteName, content, citation titles, error message) has any pre-existing envelope lines removed and is re-wrapped exactly once at the core boundary, preventing provider metadata from spoofing the marker. query always matches the requested query, citation and result URLs must parse as http(s), published must be ISO-date shaped, URLs are emitted in canonical form, and a payload containing an error key is always reported as kind: "error" with the raw provider code preserved inside the wrapped message. Raw passthrough payloads retain whatever markers the provider set.

Auto-detection

Provider lists in documentation and setup flows are arranged alphabetically. Auto-detection follows a separate, fixed precedence order and only selects a provider that requires a credential (requiresCredential !== false) when one is configured. If no provider is set, OpenClaw checks providers in this sequence and uses the first one that is ready:

API-backed providers first:

  1. Brave -- BRAVE_API_KEY or plugins.entries.brave.config.webSearch.apiKey (order 10)
  2. MiniMax Search -- MINIMAX_CODE_PLAN_KEY / MINIMAX_CODING_API_KEY / MINIMAX_OAUTH_TOKEN / MINIMAX_API_KEY or plugins.entries.minimax.config.webSearch.apiKey (order 15)
  3. Gemini -- plugins.entries.google.config.webSearch.apiKey, GEMINI_API_KEY, or models.providers.google.apiKey (order 20)
  4. Grok -- xAI OAuth, XAI_API_KEY, or plugins.entries.xai.config.webSearch.apiKey (order 30)
  5. Kimi -- KIMI_API_KEY / MOONSHOT_API_KEY or plugins.entries.moonshot.config.webSearch.apiKey (order 40)
  6. Perplexity -- PERPLEXITY_API_KEY / OPENROUTER_API_KEY or plugins.entries.perplexity.config.webSearch.apiKey (order 50)
  7. Firecrawl -- FIRECRAWL_API_KEY or plugins.entries.firecrawl.config.webSearch.apiKey (order 60)
  8. Exa -- EXA_API_KEY or plugins.entries.exa.config.webSearch.apiKey; the endpoint can be overridden with plugins.entries.exa.config.webSearch.baseUrl (optional) (order 65)
  9. Tavily -- TAVILY_API_KEY or plugins.entries.tavily.config.webSearch.apiKey (order 70)
  10. Parallel -- paid Parallel Search API via PARALLEL_API_KEY or plugins.entries.parallel.config.webSearch.apiKey; the endpoint can be overridden with plugins.entries.parallel.config.webSearch.baseUrl (optional) (order 75)

Endpoint providers configured after that:

  1. SearXNG -- SEARXNG_BASE_URL or plugins.entries.searxng.config.webSearch.baseUrl (order 200)

Providers without API keys, such as Parallel Search (Free), DuckDuckGo, Ollama Web Search, and Codex Hosted Search, are never selected by auto-detection, even though they have an internal order value. You can only use them by explicitly setting tools.web.search.provider or via openclaw configure --section web. OpenClaw will not send managed web_search queries to a key-free provider solely because no API-backed provider is configured.

OpenAI Responses models are an exception: while tools.web.search.provider is not set, they rely on OpenAI's built-in web search rather than the managed providers listed above (see below). To route them through the managed path instead, set tools.web.search.provider to parallel-free (or another provider).

Note

All provider key fields accept SecretRef objects. Plugin-scoped SecretRefs under plugins.entries.<plugin>.config.webSearch.apiKey are resolved for the installed API-backed web search providers, which include Brave, Exa, Firecrawl, Gemini, Grok, Kimi, MiniMax, Parallel, Perplexity, and Tavily. This applies whether the provider is chosen explicitly via tools.web.search.provider or through auto-detection. In auto-detect mode, OpenClaw resolves only the selected provider's key. Non-selected SecretRefs remain inactive, so you can configure multiple providers without incurring resolution cost for the ones not in use.

Direct OpenAI Responses models (api: "openai-responses", provider openai, with no base URL or an official OpenAI API base URL) automatically use OpenAI's hosted web_search tool when OpenClaw web search is enabled and no managed provider is pinned. This behavior is owned by the bundled OpenAI plugin and does not apply to OpenAI-compatible proxy base URLs or Azure routes. To keep the managed web_search tool for OpenAI models, set tools.web.search.provider to another provider such as brave. Set tools.web.search.enabled: false to disable both managed search and native OpenAI search.

The Codex app-server runtime automatically uses Codex's hosted web_search tool when web search is enabled and no managed provider is selected. Native hosted search and OpenClaw's managed web_search dynamic tool are mutually exclusive, so managed search cannot bypass native domain restrictions. OpenClaw uses the managed tool when hosted search is unavailable, explicitly disabled, or replaced by a selected managed provider. OpenClaw keeps Codex's standalone web.run extension disabled (features.standalone_web_search: false) because production app-server traffic rejects its user-defined web namespace.

  • Under tools.web.search.openaiCodex, configure native search behavior.
  • Assign tools.web.search.provider: "codex" to enable Codex Hosted Search as the managed web_search provider for any parent model. Each request triggers a short-lived ephemeral Codex app-server turn, which fails if Codex does not return a hosted webSearch item.
  • While mode: "cached" is the default, Codex resolves it to live external access for unrestricted app-server turns; set "live" to explicitly request live access.
  • To use OpenClaw's managed web_search instead, set tools.web.search.provider to a managed provider such as brave.
  • Set tools.web.search.openaiCodex.enabled: false to disable Codex-hosted search; other managed providers remain accessible.
  • Restricting the Codex native tool surface still leaves managed web_search available.
  • When allowedDomains is set, automatic managed fallback fails closed if hosted search is unavailable, preventing the native allowlist from being bypassed.
  • Tool-disabled LLM-only runs disable both native and managed search.
  • tools.web.search.enabled: false disables both managed and native search.

Persistent effective Codex search-policy changes initiate a fresh bound thread, so an already loaded app-server thread cannot retain stale hosted-search access. Transient per-turn restrictions use a temporary restricted thread and keep the existing binding for later resumption.

Direct OpenAI ChatGPT Responses traffic can also leverage OpenAI's hosted web_search tool. That separate path remains opt-in via tools.web.search.openaiCodex.enabled: true and applies only to eligible openai/* models using api: "openai-chatgpt-responses".

{
  tools: {
    web: {
      search: {
        enabled: true,
        // Optional: use Codex Hosted Search from non-Codex parent models too.
        provider: "codex",
        openaiCodex: {
          enabled: true,
          mode: "cached",
          allowedDomains: ["example.com"],
          contextSize: "high",
          userLocation: {
            country: "US",
            city: "New York",
            timezone: "America/New_York",
          },
        },
      },
    },
  },
}

For runtimes and providers that lack native Codex search support, Codex can fall back to the managed web_search through OpenClaw's dynamic tool namespace. Use an explicit managed provider when you need OpenClaw's provider-specific network controls instead of Codex-hosted search.

Selecting provider: "codex" enables the bundled codex plugin and applies the same tools.web.search.openaiCodex restrictions described above. Authenticate the Codex app-server first with openclaw models auth login --provider openai. The parent agent can use any model or runtime; only the bounded search worker runs through Codex.

Network safety

Managed HTTP web_search provider calls use OpenClaw's guarded fetch path, scoped to the current provider's own hostname. For that hostname only, OpenClaw permits Surge, Clash, and sing-box fake-IP DNS answers in 198.18.0.0/15 and fc00::/7. Other private, loopback, link-local, and metadata destinations remain blocked. Codex Hosted Search is the exception: its bounded worker delegates network access to Codex app-server's hosted web_search tool.

This automatic allowance does not extend to arbitrary web_fetch URLs. For web_fetch, explicitly enable tools.web.fetch.ssrfPolicy.allowRfc2544BenchmarkRange and tools.web.fetch.ssrfPolicy.allowIpv6UniqueLocalRange only when your trusted proxy owns those synthetic ranges.

Config

{
  tools: {
    web: {
      search: {
        enabled: true, // default: true
        provider: "brave", // or omit for auto-detection
        maxResults: 5,
        timeoutSeconds: 30,
        cacheTtlMinutes: 15,
      },
    },
  },
}

Provider-specific configuration (API keys, base URLs, modes) resides under plugins.entries.<plugin>.config.webSearch.*. Gemini can also reuse models.providers.google.apiKey and models.providers.google.baseUrl as lower-priority fallbacks after its dedicated web-search config and GEMINI_API_KEY. See the provider pages for examples. Grok can also reuse an xAI OAuth auth profile from openclaw models auth login --provider xai --method oauth; API-key config remains the fallback.

tools.web.search.provider is validated against the web-search provider IDs declared by bundled and installed plugin manifests. A typo such as "brvae" causes config validation to fail instead of silently falling back to auto-detection. If a configured provider only has stale plugin evidence, such as a leftover plugins.entries.<plugin> block after uninstalling a third-party plugin, OpenClaw keeps startup resilient and reports a warning so you can reinstall the plugin or run openclaw doctor --fix to clean up the stale config.

web_fetch fallback provider selection is separate:

  • choose it with tools.web.fetch.provider
  • or omit that field and let OpenClaw auto-detect the first ready web-fetch provider from configured credentials
  • non-sandboxed web_fetch can use installed plugin providers that declare contracts.webFetchProviders; sandboxed fetches allow bundled providers and verified official plugin installs, but exclude third-party external plugins
  • the official Firecrawl plugin is the only bundled webFetchProviders contributor today, configured under plugins.entries.firecrawl.config.webFetch.*

When you choose Kimi during openclaw onboard or openclaw configure --section web, OpenClaw can also request:

  • the Moonshot API region (https://api.moonshot.ai/v1 or https://api.moonshot.cn/v1)
  • the default Kimi web-search model (defaults to kimi-k2.6)

For x_search, set plugins.entries.xai.config.xSearch.*. This uses the same xAI authentication profile as the chat feature, or the XAI_API_KEY or plugin web-search credential that Grok web search employs. Legacy tools.web.x_search.* configuration gets migrated automatically by openclaw doctor --fix. If you select Grok during openclaw onboard or openclaw configure --section web, OpenClaw provides an optional x_search configuration step using the same credential right after Grok setup finishes. This step appears as a follow-up within the Grok flow, not as a separate top-level web-search provider option. Choosing a different provider means OpenClaw skips the x_search prompt.

Storing API keys

Config file

Execute openclaw configure --section web or directly assign the key:

{
  plugins: {
    entries: {
      brave: {
        config: {
          webSearch: {
            apiKey: "YOUR_KEY", // pragma: allowlist secret
          },
        },
      },
    },
  },
}

Environment variable

Define the provider environment variable within the Gateway process environment:

export BRAVE_API_KEY="YOUR_KEY"

For a gateway installation, place it inside ~/.openclaw/.env.
Refer to Env vars.

Tool parameters

ParameterDescription
querySearch query (required)
countResults to return (1-10, default: 5)
country2-letter ISO country code (e.g. "US", "DE")
languageISO 639-1 language code (e.g. "en", "de")
search_langSearch-language code (Brave only)
freshnessTime filter: day, week, month, or year
date_afterResults after this date (YYYY-MM-DD)
date_beforeResults before this date (YYYY-MM-DD)
ui_langUI language code (Brave only)
domain_filterDomain allowlist/denylist array (Perplexity only)
max_tokensTotal content token budget, native Perplexity Search API only
max_tokens_per_pagePer-page extraction token limit, native Perplexity Search API only

Warning

Not every parameter works across all providers. Brave llm-context mode rejects ui_lang; date_before also requires date_after because Brave custom freshness ranges demand both start and end dates. Gemini, Grok, and Kimi each return a single synthesized answer with citations. They accept count for shared-tool compatibility, but it does not alter the grounded answer format. Gemini treats day freshness as a recency hint; broader freshness values and explicit dates set Google Search grounding time ranges. Perplexity behaves identically when using the Sonar/OpenRouter compatibility path (plugins.entries.perplexity.config.webSearch.baseUrl / model or OPENROUTER_API_KEY); that path also drops max_tokens and max_tokens_per_page support. SearXNG accepts http:// only for trusted private-network or loopback hosts; public SearXNG endpoints must use https://. Firecrawl and Tavily only support query and count through web_search -- use their dedicated tools for advanced options.

x_search searches X (formerly Twitter) posts through xAI and returns AI-generated answers with citations. It accepts natural-language queries along with optional structured filters. OpenClaw builds the built-in xAI x_search tool per request instead of keeping it permanently registered, so it remains active only for the turn that invokes it.

Warning

x_search executes on xAI's servers. xAI charges $5 per 1,000 tool calls, plus the model's input and output tokens.

Note

xAI documents x_search as supporting keyword search, semantic search, user search, and thread fetch. For per-post engagement metrics like reposts, replies, bookmarks, or views, prefer a targeted lookup for the exact post URL or status ID. Broad keyword searches may locate the right post but return less complete per-post metadata. A recommended approach is: find the post first, then run a second x_search query focused on that specific post.

x_search config

When enabled is left out, x_search appears only if the active model uses xai as its provider and xAI credentials are successfully resolved. For an active model whose provider is known but not xAI, assign plugins.entries.xai.config.xSearch.enabled the value true to enable cross-provider usage. If the active model's provider is absent or cannot be determined, the tool remains hidden. Set enabled to false to turn it off for all providers. xAI credentials are mandatory in every case.

{
  plugins: {
    entries: {
      xai: {
        config: {
          xSearch: {
            enabled: true, // required for a known non-xAI model provider
            model: "grok-4.3",
            baseUrl: "https://api.x.ai/v1", // optional, overrides webSearch.baseUrl
            inlineCitations: false,
            maxTurns: 2,
            timeoutSeconds: 30,
            cacheTtlMinutes: 15,
          },
          webSearch: {
            apiKey: "xai-...", // optional if an xAI auth profile or XAI_API_KEY is set
            baseUrl: "https://api.x.ai/v1", // optional shared xAI Responses base URL
          },
        },
      },
    },
  },
}

x_search sends data to <baseUrl>/responses whenever plugins.entries.xai.config.xSearch.baseUrl is configured. When that field is not provided, it defaults to plugins.entries.xai.config.webSearch.baseUrl and then to the public xAI endpoint at https://api.x.ai/v1.

x_search parameters

ParameterDescription
querySearch query (required)
allowed_x_handlesLimit returned results to a maximum of 20 X handles
excluded_x_handlesOmit up to 20 X handles from results
from_dateInclude only posts from this date onward (YYYY-MM-DD)
to_dateInclude only posts up to this date (YYYY-MM-DD)
enable_image_understandingAllow xAI to examine images attached to matching posts
enable_video_understandingAllow xAI to examine videos attached to matching posts

allowed_x_handles and excluded_x_handles cannot be used together.

x_search example

await x_search({
  query: "dinner recipes",
  allowed_x_handles: ["nytfood"],
  from_date: "2026-03-01",
});
// Per-post stats: use the exact status URL or status ID when possible
await x_search({
  query: "https://x.com/huntharo/status/1905678901234567890",
});

Examples

// Basic search
await web_search({ query: "OpenClaw plugin SDK" });

// German-specific search
await web_search({ query: "TV online schauen", country: "DE", language: "de" });

// Recent results (past week)
await web_search({ query: "AI developments", freshness: "week" });

// Date range
await web_search({
  query: "climate research",
  date_after: "2024-01-01",
  date_before: "2024-06-30",
});

// Domain filtering (Perplexity only)
await web_search({
  query: "product reviews",
  domain_filter: ["-reddit.com", "-pinterest.com"],
});

Tool profiles

When using tool profiles or allowlists, include web_search, x_search, or group:web:

{
  tools: {
    allow: ["web_search", "x_search"],
    // or: allow: ["group:web"]  (includes web_search, x_search, and web_fetch)
  },
}