Web Search, X Search, and Web Fetch Tools
Learn how to use web_search, x_search, and web_fetch for web searches, X content retrieval, and URL fetching. This guide covers provider selection, configuration, and usage for developers.
Read this when
- You want to enable or configure web_search
- You want to enable or configure x_search
- You need to choose a search provider
- You want to understand auto-detection and provider selection
web_search performs web searches through your chosen provider and delivers consistent results, with query caching lasting 15 minutes (adjustable). OpenClaw additionally includes x_search for X (previously Twitter) content and web_fetch for simple URL retrieval. web_fetch operates entirely on your machine; web_search goes through xAI Responses when Grok serves as the provider, and x_search consistently uses xAI Responses.
Info
web_searchis a basic HTTP tool, not a browser. For sites that rely heavily on JavaScript or require login credentials, turn to the Web Browser. To retrieve a particular URL, use Web Fetch.
Quick start
Choose a provider
Select a provider and finish any necessary configuration. Certain providers require no key, while others demand an API key. Consult the individual provider pages for specifics.
Configure
openclaw configure --section web
This command saves the provider and any required credentials. For providers backed by an API, you can alternatively set the provider's environment variable (such as BRAVE_API_KEY) and omit this step.
Use it
await web_search({ query: "OpenClaw plugin SDK" });
For X posts:
await x_search({ query: "dinner recipes" });
Choosing a provider
-
Brave Search, Organized results that include excerpts. Offers
llm-contextmode along with country and language filters. A free tier exists. -
Codex Hosted Search, AI-generated grounded responses using your Codex app-server account.
-
DuckDuckGo, No key required. No API key necessary. Uses an unofficial HTML-based integration.
-
Exa, Combines neural and keyword search with content extraction features such as highlights, text, and summaries.
-
Firecrawl, Delivers structured results. Works optimally when combined with
firecrawl_searchandfirecrawl_scrapefor deeper extraction. -
Gemini, AI-generated answers with citations, powered by Google Search grounding.
-
Grok, AI-generated answers with citations, powered by xAI web grounding.
-
Kimi, AI-generated answers with citations, powered by Moonshot web search; ungrounded chat fallbacks fail with an error.
-
MiniMax Search, Structured results retrieved through the MiniMax Token Plan search API.
-
Ollama Web Search, Performs searches via a signed-in local Ollama instance or the hosted Ollama API.
-
Parallel, A paid Parallel Search API (
PARALLEL_API_KEY) offering higher rate limits and objective tuning. -
Parallel Search (Free), A key-free opt-in option. Uses Parallel's free Search MCP, providing LLM-optimized dense excerpts without requiring an API key.
-
Perplexity, Structured results with controls for content extraction and domain filtering.
-
SearXNG, A self-hosted meta-search engine. No API key needed. Gathers results from Google, Bing, DuckDuckGo, and others.
-
Tavily, Structured results featuring search depth, topic filtering, and
tavily_extractfor URL extraction.
Provider comparison
| Provider | Result style | Filters | API key |
|---|---|---|---|
| Brave | Structured snippets | Country, language, time, llm-context mode | BRAVE_API_KEY |
| Codex Hosted Search | AI-synthesized + source URLs | Domains, context size, user location | None; uses Codex/OpenAI sign-in |
| DuckDuckGo | Structured snippets | -- | None (key-free) |
| Exa | Structured + extracted | Neural/keyword mode, date, content extraction | EXA_API_KEY |
| Firecrawl | Structured snippets | Via firecrawl_search tool | FIRECRAWL_API_KEY |
| Gemini | AI-synthesized + citations | -- | GEMINI_API_KEY |
| Grok | AI-synthesized + citations | -- | xAI OAuth, XAI_API_KEY, or plugins.entries.xai.config.webSearch.apiKey |
| Kimi | AI-synthesized + citations; fails on ungrounded chat fallbacks | -- | KIMI_API_KEY / MOONSHOT_API_KEY |
| MiniMax Search | Structured snippets | Region (global / cn) | MINIMAX_CODE_PLAN_KEY / MINIMAX_CODING_API_KEY / MINIMAX_OAUTH_TOKEN |
| Ollama Web Search | Structured snippets | -- | None for signed-in local hosts; OLLAMA_API_KEY for direct https://ollama.com search |
| Parallel | Dense excerpts ranked for LLM context | -- | PARALLEL_API_KEY (paid) |
| Parallel Search (Free) | Dense excerpts ranked for LLM context | -- | None (free Search MCP) |
| Perplexity | Structured snippets | Country, language, time, domains, content limits | PERPLEXITY_API_KEY / OPENROUTER_API_KEY |
| SearXNG | Structured snippets | Categories, language | None (self-hosted) |
| Tavily | Structured snippets | Via tavily_search tool | TAVILY_API_KEY |
Result shape
At the core tool boundary, web_search standardizes every bundled and external plugin provider. Callers always receive one of these closed shapes:
type WebSearchOutput =
| {
kind: "error";
provider: string;
error: "provider_error";
message: string;
docs?: string;
}
| {
kind: "results";
provider: string;
query: string;
count: number;
tookMs?: number;
results: Array<{
title: string;
url: string;
snippet?: string;
published?: string;
siteName?: string;
}>;
externalContent: {
untrusted: true;
source: "web_search";
wrapped: true;
provider: string;
};
cached?: true;
}
| {
kind: "answer";
provider: string;
query: string;
tookMs?: number;
content: string;
citations?: Array<{ url: string; title?: string }>;
externalContent: {
untrusted: true;
source: "web_search";
wrapped: true;
provider: string;
};
cached?: true;
}
| {
kind: "raw";
provider: string;
data: unknown;
};
Structured providers employ kind: "results", while synthesized providers use kind: "answer". For compatibility, external plugin providers whose payloads do not match either shape pass through unchanged as kind: "raw". Provider-specific details like raw scores, excerpts, related searches, inline-citation offsets, model ids, or session metadata are not forwarded on normalized branches. When a provider's richer response is part of your workflow, use its dedicated tool.
externalContent.wrapped: true acts as a trust marker that the boundary itself guarantees to be true. Provider prose (title, snippet, siteName, content, citation titles, error message) has any pre-existing envelope lines removed and is re-wrapped exactly once at the core boundary, preventing provider metadata from spoofing the marker. query always matches the requested query, citation and result URLs must parse as http(s), published must be ISO-date shaped, URLs are emitted in canonical form, and a payload containing an error key is always reported as kind: "error" with the raw provider code preserved inside the wrapped message. Raw passthrough payloads retain whatever markers the provider set.
Auto-detection
Provider lists in documentation and setup flows are arranged alphabetically. Auto-detection follows a separate, fixed precedence order and only selects a provider that requires a credential (requiresCredential !== false) when one is configured. If no provider is set, OpenClaw checks providers in this sequence and uses the first one that is ready:
API-backed providers first:
- Brave --
BRAVE_API_KEYorplugins.entries.brave.config.webSearch.apiKey(order 10) - MiniMax Search --
MINIMAX_CODE_PLAN_KEY/MINIMAX_CODING_API_KEY/MINIMAX_OAUTH_TOKEN/MINIMAX_API_KEYorplugins.entries.minimax.config.webSearch.apiKey(order 15) - Gemini --
plugins.entries.google.config.webSearch.apiKey,GEMINI_API_KEY, ormodels.providers.google.apiKey(order 20) - Grok -- xAI OAuth,
XAI_API_KEY, orplugins.entries.xai.config.webSearch.apiKey(order 30) - Kimi --
KIMI_API_KEY/MOONSHOT_API_KEYorplugins.entries.moonshot.config.webSearch.apiKey(order 40) - Perplexity --
PERPLEXITY_API_KEY/OPENROUTER_API_KEYorplugins.entries.perplexity.config.webSearch.apiKey(order 50) - Firecrawl --
FIRECRAWL_API_KEYorplugins.entries.firecrawl.config.webSearch.apiKey(order 60) - Exa --
EXA_API_KEYorplugins.entries.exa.config.webSearch.apiKey; the endpoint can be overridden withplugins.entries.exa.config.webSearch.baseUrl(optional) (order 65) - Tavily --
TAVILY_API_KEYorplugins.entries.tavily.config.webSearch.apiKey(order 70) - Parallel -- paid Parallel Search API via
PARALLEL_API_KEYorplugins.entries.parallel.config.webSearch.apiKey; the endpoint can be overridden withplugins.entries.parallel.config.webSearch.baseUrl(optional) (order 75)
Endpoint providers configured after that:
- SearXNG --
SEARXNG_BASE_URLorplugins.entries.searxng.config.webSearch.baseUrl(order 200)
Providers without API keys, such as Parallel Search (Free), DuckDuckGo,
Ollama Web Search, and Codex Hosted Search, are never selected by
auto-detection, even though they have an internal order value. You can only use
them by explicitly setting tools.web.search.provider or via
openclaw configure --section web. OpenClaw will not send managed
web_search queries to a key-free provider solely because no API-backed
provider is configured.
OpenAI Responses models are an exception: while tools.web.search.provider
is not set, they rely on OpenAI's built-in web search rather than the managed
providers listed above (see below). To route them through the managed path
instead, set tools.web.search.provider to
parallel-free (or another provider).
Note
All provider key fields accept SecretRef objects. Plugin-scoped SecretRefs under
plugins.entries.<plugin>.config.webSearch.apiKeyare resolved for the installed API-backed web search providers, which include Brave, Exa, Firecrawl, Gemini, Grok, Kimi, MiniMax, Parallel, Perplexity, and Tavily. This applies whether the provider is chosen explicitly viatools.web.search.provideror through auto-detection. In auto-detect mode, OpenClaw resolves only the selected provider's key. Non-selected SecretRefs remain inactive, so you can configure multiple providers without incurring resolution cost for the ones not in use.
Native OpenAI web search
Direct OpenAI Responses models (api: "openai-responses", provider openai,
with no base URL or an official OpenAI API base URL) automatically use OpenAI's
hosted web_search tool when OpenClaw web search is enabled and no managed
provider is pinned. This behavior is owned by the bundled OpenAI plugin and does
not apply to OpenAI-compatible proxy base URLs or Azure routes. To keep the
managed web_search tool for OpenAI models, set
tools.web.search.provider to another provider such as brave. Set
tools.web.search.enabled: false to disable both managed search and native
OpenAI search.
Native Codex web search
The Codex app-server runtime automatically uses Codex's hosted
web_search tool when web search is enabled and no managed provider is
selected. Native hosted search and OpenClaw's managed
web_search dynamic tool are mutually exclusive, so managed search cannot
bypass native domain restrictions. OpenClaw uses the managed tool when hosted
search is unavailable, explicitly disabled, or replaced by a selected managed
provider. OpenClaw keeps Codex's standalone
web.run extension disabled (features.standalone_web_search: false)
because production app-server traffic rejects its user-defined
web namespace.
- Under
tools.web.search.openaiCodex, configure native search behavior. - Assign
tools.web.search.provider: "codex"to enable Codex Hosted Search as the managedweb_searchprovider for any parent model. Each request triggers a short-lived ephemeral Codex app-server turn, which fails if Codex does not return a hostedwebSearchitem. - While
mode: "cached"is the default, Codex resolves it to live external access for unrestricted app-server turns; set"live"to explicitly request live access. - To use OpenClaw's managed
web_searchinstead, settools.web.search.providerto a managed provider such asbrave. - Set
tools.web.search.openaiCodex.enabled: falseto disable Codex-hosted search; other managed providers remain accessible. - Restricting the Codex native tool surface still leaves managed
web_searchavailable. - When
allowedDomainsis set, automatic managed fallback fails closed if hosted search is unavailable, preventing the native allowlist from being bypassed. - Tool-disabled LLM-only runs disable both native and managed search.
tools.web.search.enabled: falsedisables both managed and native search.
Persistent effective Codex search-policy changes initiate a fresh bound thread, so an already loaded app-server thread cannot retain stale hosted-search access. Transient per-turn restrictions use a temporary restricted thread and keep the existing binding for later resumption.
Direct OpenAI ChatGPT Responses traffic can also leverage OpenAI's hosted web_search tool. That separate path remains opt-in via tools.web.search.openaiCodex.enabled: true and applies only to eligible openai/* models using api: "openai-chatgpt-responses".
{
tools: {
web: {
search: {
enabled: true,
// Optional: use Codex Hosted Search from non-Codex parent models too.
provider: "codex",
openaiCodex: {
enabled: true,
mode: "cached",
allowedDomains: ["example.com"],
contextSize: "high",
userLocation: {
country: "US",
city: "New York",
timezone: "America/New_York",
},
},
},
},
},
}
For runtimes and providers that lack native Codex search support, Codex can fall back to the managed web_search through OpenClaw's dynamic tool namespace. Use an explicit managed provider when you need OpenClaw's provider-specific network controls instead of Codex-hosted search.
Selecting provider: "codex" enables the bundled codex plugin and applies the same tools.web.search.openaiCodex restrictions described above. Authenticate the Codex app-server first with openclaw models auth login --provider openai. The parent agent can use any model or runtime; only the bounded search worker runs through Codex.
Network safety
Managed HTTP web_search provider calls use OpenClaw's guarded fetch path, scoped to the current provider's own hostname. For that hostname only, OpenClaw permits Surge, Clash, and sing-box fake-IP DNS answers in 198.18.0.0/15 and fc00::/7. Other private, loopback, link-local, and metadata destinations remain blocked. Codex Hosted Search is the exception: its bounded worker delegates network access to Codex app-server's hosted web_search tool.
This automatic allowance does not extend to arbitrary web_fetch URLs. For web_fetch, explicitly enable tools.web.fetch.ssrfPolicy.allowRfc2544BenchmarkRange and tools.web.fetch.ssrfPolicy.allowIpv6UniqueLocalRange only when your trusted proxy owns those synthetic ranges.
Config
{
tools: {
web: {
search: {
enabled: true, // default: true
provider: "brave", // or omit for auto-detection
maxResults: 5,
timeoutSeconds: 30,
cacheTtlMinutes: 15,
},
},
},
}
Provider-specific configuration (API keys, base URLs, modes) resides under plugins.entries.<plugin>.config.webSearch.*. Gemini can also reuse models.providers.google.apiKey and models.providers.google.baseUrl as lower-priority fallbacks after its dedicated web-search config and GEMINI_API_KEY. See the provider pages for examples. Grok can also reuse an xAI OAuth auth profile from openclaw models auth login --provider xai --method oauth; API-key config remains the fallback.
tools.web.search.provider is validated against the web-search provider IDs declared by bundled and installed plugin manifests. A typo such as "brvae" causes config validation to fail instead of silently falling back to auto-detection. If a configured provider only has stale plugin evidence, such as a leftover plugins.entries.<plugin> block after uninstalling a third-party plugin, OpenClaw keeps startup resilient and reports a warning so you can reinstall the plugin or run openclaw doctor --fix to clean up the stale config.
web_fetch fallback provider selection is separate:
- choose it with
tools.web.fetch.provider - or omit that field and let OpenClaw auto-detect the first ready web-fetch provider from configured credentials
- non-sandboxed
web_fetchcan use installed plugin providers that declarecontracts.webFetchProviders; sandboxed fetches allow bundled providers and verified official plugin installs, but exclude third-party external plugins - the official Firecrawl plugin is the only bundled
webFetchProviderscontributor today, configured underplugins.entries.firecrawl.config.webFetch.*
When you choose Kimi during openclaw onboard or openclaw configure --section web, OpenClaw can also request:
- the Moonshot API region (
https://api.moonshot.ai/v1orhttps://api.moonshot.cn/v1) - the default Kimi web-search model (defaults to
kimi-k2.6)
For x_search, set plugins.entries.xai.config.xSearch.*. This uses the same xAI authentication profile as the chat feature, or the XAI_API_KEY or plugin web-search credential that Grok web search employs. Legacy tools.web.x_search.* configuration gets migrated automatically by openclaw doctor --fix. If you select Grok during openclaw onboard or openclaw configure --section web, OpenClaw provides an optional x_search configuration step using the same credential right after Grok setup finishes. This step appears as a follow-up within the Grok flow, not as a separate top-level web-search provider option. Choosing a different provider means OpenClaw skips the x_search prompt.
Storing API keys
Config file
Execute openclaw configure --section web or directly assign the key:
{
plugins: {
entries: {
brave: {
config: {
webSearch: {
apiKey: "YOUR_KEY", // pragma: allowlist secret
},
},
},
},
},
}
Environment variable
Define the provider environment variable within the Gateway process environment:
export BRAVE_API_KEY="YOUR_KEY"
For a gateway installation, place it inside ~/.openclaw/.env.
Refer to Env vars.
Tool parameters
| Parameter | Description |
|---|---|
query | Search query (required) |
count | Results to return (1-10, default: 5) |
country | 2-letter ISO country code (e.g. "US", "DE") |
language | ISO 639-1 language code (e.g. "en", "de") |
search_lang | Search-language code (Brave only) |
freshness | Time filter: day, week, month, or year |
date_after | Results after this date (YYYY-MM-DD) |
date_before | Results before this date (YYYY-MM-DD) |
ui_lang | UI language code (Brave only) |
domain_filter | Domain allowlist/denylist array (Perplexity only) |
max_tokens | Total content token budget, native Perplexity Search API only |
max_tokens_per_page | Per-page extraction token limit, native Perplexity Search API only |
Warning
Not every parameter works across all providers. Brave
llm-contextmode rejectsui_lang;date_beforealso requiresdate_afterbecause Brave custom freshness ranges demand both start and end dates. Gemini, Grok, and Kimi each return a single synthesized answer with citations. They acceptcountfor shared-tool compatibility, but it does not alter the grounded answer format. Gemini treatsdayfreshness as a recency hint; broader freshness values and explicit dates set Google Search grounding time ranges. Perplexity behaves identically when using the Sonar/OpenRouter compatibility path (plugins.entries.perplexity.config.webSearch.baseUrl/modelorOPENROUTER_API_KEY); that path also dropsmax_tokensandmax_tokens_per_pagesupport. SearXNG acceptshttp://only for trusted private-network or loopback hosts; public SearXNG endpoints must usehttps://. Firecrawl and Tavily only supportqueryandcountthroughweb_search-- use their dedicated tools for advanced options.
x_search
x_search searches X (formerly Twitter) posts through xAI and returns AI-generated answers with citations. It accepts natural-language queries along with optional structured filters. OpenClaw builds the built-in xAI x_search tool per request instead of keeping it permanently registered, so it remains active only for the turn that invokes it.
Warning
x_searchexecutes on xAI's servers. xAI charges $5 per 1,000 tool calls, plus the model's input and output tokens.
Note
xAI documents
x_searchas supporting keyword search, semantic search, user search, and thread fetch. For per-post engagement metrics like reposts, replies, bookmarks, or views, prefer a targeted lookup for the exact post URL or status ID. Broad keyword searches may locate the right post but return less complete per-post metadata. A recommended approach is: find the post first, then run a secondx_searchquery focused on that specific post.
x_search config
When enabled is left out, x_search appears only if the active model uses xai as its provider and xAI credentials are successfully resolved. For an active model whose provider is known but not xAI, assign plugins.entries.xai.config.xSearch.enabled the value true to enable cross-provider usage. If the active model's provider is absent or cannot be determined, the tool remains hidden. Set enabled to false to turn it off for all providers. xAI credentials are mandatory in every case.
{
plugins: {
entries: {
xai: {
config: {
xSearch: {
enabled: true, // required for a known non-xAI model provider
model: "grok-4.3",
baseUrl: "https://api.x.ai/v1", // optional, overrides webSearch.baseUrl
inlineCitations: false,
maxTurns: 2,
timeoutSeconds: 30,
cacheTtlMinutes: 15,
},
webSearch: {
apiKey: "xai-...", // optional if an xAI auth profile or XAI_API_KEY is set
baseUrl: "https://api.x.ai/v1", // optional shared xAI Responses base URL
},
},
},
},
},
}
x_search sends data to <baseUrl>/responses whenever plugins.entries.xai.config.xSearch.baseUrl is configured. When that field is not provided, it defaults to plugins.entries.xai.config.webSearch.baseUrl and then to the public xAI endpoint at https://api.x.ai/v1.
x_search parameters
| Parameter | Description |
|---|---|
query | Search query (required) |
allowed_x_handles | Limit returned results to a maximum of 20 X handles |
excluded_x_handles | Omit up to 20 X handles from results |
from_date | Include only posts from this date onward (YYYY-MM-DD) |
to_date | Include only posts up to this date (YYYY-MM-DD) |
enable_image_understanding | Allow xAI to examine images attached to matching posts |
enable_video_understanding | Allow xAI to examine videos attached to matching posts |
allowed_x_handles and excluded_x_handles cannot be used together.
x_search example
await x_search({
query: "dinner recipes",
allowed_x_handles: ["nytfood"],
from_date: "2026-03-01",
});
// Per-post stats: use the exact status URL or status ID when possible
await x_search({
query: "https://x.com/huntharo/status/1905678901234567890",
});
Examples
// Basic search
await web_search({ query: "OpenClaw plugin SDK" });
// German-specific search
await web_search({ query: "TV online schauen", country: "DE", language: "de" });
// Recent results (past week)
await web_search({ query: "AI developments", freshness: "week" });
// Date range
await web_search({
query: "climate research",
date_after: "2024-01-01",
date_before: "2024-06-30",
});
// Domain filtering (Perplexity only)
await web_search({
query: "product reviews",
domain_filter: ["-reddit.com", "-pinterest.com"],
});
Tool profiles
When using tool profiles or allowlists, include web_search, x_search, or group:web:
{
tools: {
allow: ["web_search", "x_search"],
// or: allow: ["group:web"] (includes web_search, x_search, and web_fetch)
},
}
Related
- Web Fetch -- retrieve a URL and extract its readable content
- Web Browser -- full browser automation for JavaScript-heavy pages
- Grok Search -- Grok as the
web_searchprovider - Ollama Web Search -- key-free web search using your Ollama host