Prompt caching: knobs, merge order, provider behavior, and tuning patterns

Prompt caching allows a model provider to reuse an unchanged prompt prefix (system or developer instructions, tool definitions, and other stable context) across multiple turns, rather than reprocessing it with each request. This reduces token costs and latency during long-running sessions that repeatedly send the same context.

OpenClaw normalizes provider usage into cacheRead and cacheWrite whenever the upstream API exposes those counters. Usage summaries (/status and similar) default to the last transcript usage entry when the live session snapshot lacks cache counters; a nonzero live value always takes precedence over the fallback.

Provider references:

• Anthropic prompt caching
• OpenAI prompt caching

Primary knobs

cacheRetention

Values: "none" | "short" | "long". Can be set as a global default, per model, and per agent. "standard" is not an alias; use "short" to apply the provider's default cache window. Invalid values are ignored and a warning is emitted.

agents:
  defaults:
    params:
      cacheRetention: "long" # none | short | long
    models:
      "anthropic/claude-opus-4-6":
        params:
          cacheRetention: "short" # overrides the global default for this model
  list:
    - id: "alerts"
      params:
        cacheRetention: "none" # overrides both defaults for this agent

Merge order (later wins):

1. agents.defaults.params - global default for all models
2. agents.defaults.models["provider/model"].params - per-model override
3. agents.entries.*.params - per-agent override, matched by agent id

Source: src/agents/embedded-agent-runner/extra-params.ts (resolveExtraParams).

contextPruning.mode: "cache-ttl"

After the cache TTL window expires, old tool-result context is pruned so a post-idle request does not recache an oversized history.

agents:
  defaults:
    contextPruning:
      mode: "cache-ttl"
      ttl: "1h"

See Session pruning for full behavior.

Heartbeat keep-warm

A heartbeat can keep cache windows warm and reduce repeated cache writes after idle gaps. Configurable globally (agents.defaults.heartbeat) or per agent (agents.entries.*.heartbeat).

agents:
  defaults:
    heartbeat:
      every: "55m"

Provider behavior

Anthropic (direct API and Vertex AI)

• cacheRetention is supported for anthropic and anthropic-vertex providers, and for Claude models on amazon-bedrock and custom anthropic-messages-compatible endpoints when cacheRetention is set explicitly.
• When unset, OpenClaw seeds cacheRetention: "short" for direct Anthropic (anthropic and anthropic-vertex providers only; other Anthropic-family routes require an explicit value).
• Native Anthropic Messages responses expose cache_read_input_tokens and cache_creation_input_tokens, mapped to cacheRead and cacheWrite.
• cacheRetention: "short" maps to the default 5-minute ephemeral cache. cacheRetention: "long" requests the 1-hour TTL (cache_control: { type: "ephemeral", ttl: "1h" }) when set explicitly. An implicit or env-driven long retention (OPENCLAW_CACHE_RETENTION=long with no explicit cacheRetention) only upgrades to the 1-hour TTL on api.anthropic.com or Vertex AI (aiplatform.googleapis.com / *-aiplatform.googleapis.com) hosts; other hosts keep the 5-minute cache.

Source: packages/ai/src/transports/anthropic-payload-policy.ts (resolveAnthropicEphemeralCacheControl, isLongTtlEligibleEndpoint).

OpenAI (direct API)

• For supported recent models, prompt caching happens automatically. OpenClaw does not insert any block-level cache markers.
• OpenClaw sends prompt_cache_key to maintain stable cache routing across turns. Direct api.openai.com hosts receive this automatically. OpenAI-compatible proxies (oMLX, llama.cpp, custom endpoints) require compat.supportsPromptCacheKey: true in the model configuration to opt in; this is never auto-detected for a proxy.
• prompt_cache_retention: "24h" is only added when cacheRetention: "long" is selected and the resolved endpoint supports both the cache key and long retention (compat.supportsLongCacheRetention, enabled by default; Together AI and Cloudflare compatibility profiles disable it). cacheRetention: "none" suppresses both fields.
• Cache hits are exposed via usage.prompt_tokens_details.cached_tokens (Chat Completions) or input_tokens_details.cached_tokens (Responses API), mapped to cacheRead.
• Responses API payloads can also expose input_tokens_details.cache_write_tokens, mapped to cacheWrite and billed at the model's cache-write rate. Responses payloads that omit the field keep cacheWrite at 0. OpenAI's Chat Completions API does not document or emit a cache_write_tokens counter, but OpenClaw still reads prompt_tokens_details.cache_write_tokens there for OpenRouter-compatible and DeepSeek-style proxies that report a separate write count.
• In practice, OpenAI behaves more like an initial-prefix cache than Anthropic's moving full-history reuse. See OpenAI live expectations below.

Amazon Bedrock

• Anthropic Claude model refs (amazon-bedrock/*anthropic.claude*, plus AWS system inference profile prefixes us./eu./global.anthropic.claude*) support explicit cacheRetention pass-through.
• Non-Anthropic Bedrock models (for example amazon.nova-*) resolve to no cache retention at runtime, regardless of any configured cacheRetention value.
• Opaque Bedrock application inference profile ARNs (profile IDs that do not contain claude) also resolve to no cache retention unless cacheRetention is set explicitly, since the model family cannot be inferred from the ARN alone.

OpenRouter

For openrouter/anthropic/* model refs, OpenClaw injects Anthropic cache_control markers on system/developer prompt blocks, but only when the request still targets a verified OpenRouter route (openrouter on its default endpoint, or any provider/base URL that resolves to openrouter.ai). Repointing the model at an arbitrary OpenAI-compatible proxy URL stops this injection.

contextPruning.mode: "cache-ttl" is allowed for openrouter/anthropic/*, openrouter/deepseek/*, openrouter/moonshot/*, openrouter/moonshotai/*, and openrouter/zai/* model refs, because these routes handle provider-side prompt caching without needing OpenClaw's injected markers.

Source: extensions/openrouter/index.ts (OPENROUTER_CACHE_TTL_MODEL_PREFIXES).

DeepSeek cache construction on OpenRouter is best-effort and can take a few seconds. An immediate follow-up request may still show cached_tokens: 0. Verify with a repeated same-prefix request after a short delay, using usage.prompt_tokens_details.cached_tokens as the cache-hit signal.

Google Gemini (direct API)

• Direct Gemini transport (api: "google-generative-ai") reports cache hits through upstream cachedContentTokenCount, mapped to cacheRead.
• Eligible model families: gemini-2.5* and gemini-3* (excludes Live/preview variants outside that prefix match, for example gemini-live-2.5-flash-preview).
• When cacheRetention is set on an eligible model, OpenClaw automatically creates, reuses, and refreshes a cachedContents resource for the system prompt. No manual cached-content handle is needed. TTL is 300s for cacheRetention: "short" and 3600s for "long".
• You can still pass a pre-existing Gemini cached-content handle through as params.cachedContent (or legacy params.cached_content). An explicit handle skips the automatic cache-management path entirely.
• This is separate from Anthropic/OpenAI prompt-prefix caching. OpenClaw manages a provider-native cachedContents resource for Gemini instead of injecting inline cache markers.

Source: src/agents/embedded-agent-runner/google-prompt-cache.ts.

CLI-harness providers (Claude Code, Gemini CLI)

CLI backends that produce JSONL usage events (jsonlDialect: "claude-stream-json" or "gemini-stream-json") are processed by a shared usage parser that accepts multiple field-name variations, including a bare cached counter associated with cacheRead. If the CLI's JSON payload does not include an explicit input-token field, OpenClaw calculates it as input_tokens - cached. This is strictly usage normalization; it does not generate Anthropic/OpenAI-style prompt-cache markers for CLI-driven models.

Source: src/agents/cli-output.ts (toCliUsage).

Other providers

When a provider does not support any of the cache modes listed above, cacheRetention has no effect.

System-prompt cache boundary

OpenClaw divides the system prompt into a stable prefix and a volatile suffix using an internal cache-prefix boundary. Content placed above the boundary (tool definitions, skills metadata, workspace files) is arranged to remain byte-identical between turns. Content below the boundary (for example HEARTBEAT.md, runtime timestamps, and other per-turn metadata) can vary without breaking the cached prefix.

Key design decisions:

• Stable workspace project-context files are positioned before HEARTBEAT.md so heartbeat churn does not disrupt the stable prefix.
• The boundary applies across Anthropic-family, OpenAI-family, Google, and CLI transport shaping, so every supported provider gains the same prefix stability.
• Codex Responses and Anthropic Vertex requests are processed through boundary-aware cache shaping so cache reuse aligns with what providers actually receive.
• System-prompt fingerprints are normalized (whitespace, line endings, hook-added context, runtime capability ordering) so semantically identical prompts share cache across turns.

If you notice unexpected cacheWrite spikes following a configuration or workspace change, determine whether the change falls above or below the cache boundary. Moving volatile content below the boundary (or stabilizing it) typically resolves the problem.

OpenClaw cache-stability guards

• Bundled MCP tool catalogs are sorted deterministically (by server name, then tool name) before tool registration, so changes in listTools() order do not churn the tools block and break prompt-cache prefixes.
• Legacy sessions containing persisted image blocks keep the 3 most recent completed turns intact (counting all completed turns, not only those with images). Older already-processed image blocks are replaced with a text marker so image-heavy follow-ups do not repeatedly send large stale payloads.

Tuning patterns

Mixed traffic (recommended default)

Maintain a long-lived baseline on your primary agent, and disable caching on bursty notifier agents:

agents:
  defaults:
    model:
      primary: "anthropic/claude-opus-4-6"
    models:
      "anthropic/claude-opus-4-6":
        params:
          cacheRetention: "long"
  list:
    - id: "research"
      default: true
      heartbeat:
        every: "55m"
    - id: "alerts"
      params:
        cacheRetention: "none"

Cost-first baseline

• Set baseline cacheRetention: "short".
• Enable contextPruning.mode: "cache-ttl".
• Keep heartbeat below your TTL only for agents that benefit from warm caches.

Live regression tests

OpenClaw runs a single combined live cache regression gate that covers repeated prefixes, tool turns, image turns, MCP-style tool transcripts, and an Anthropic no-cache control.

• src/agents/live-cache-regression.live.test.ts
• src/agents/live-cache-regression-runner.ts
• src/agents/live-cache-regression-baseline.ts

Execute it with:

OPENCLAW_LIVE_TEST=1 OPENCLAW_LIVE_CACHE_TEST=1 pnpm test:live:cache

The baseline file stores the most recently observed live numbers along with the provider-specific regression floors the test validates against. Each run uses fresh per-run session IDs and prompt namespaces so previous cache state does not contaminate the current sample. Anthropic and OpenAI enforce differently: an Anthropic floor miss is a hard regression (the test fails), while an OpenAI floor miss is watch-only (logged as a warning, does not fail the run). No single cross-provider threshold is shared.

Anthropic live expectations

• Expect explicit warmup writes through cacheWrite.
• Expect near-full history reuse on repeated turns, because Anthropic's cache control advances the cache breakpoint through the conversation.
• Baseline floors for stable, tool, image, and MCP-style lanes are hard regression gates.

OpenAI live expectations

• Expect cacheRead only; cacheWrite remains 0 on Chat Completions.
• Treat repeated-turn cache reuse as a provider-specific plateau, not Anthropic-style moving full-history reuse.
• Floors are watch-only (a miss is logged as a warning, not a test failure), derived from observed live behavior on gpt-5.4-mini:

The most recently observed baseline numbers (from live-cache-regression-baseline.ts) landed at: stable prefix cacheRead=4864, hit rate 0.966; tool transcript cacheRead=4608, hit rate 0.896; image transcript cacheRead=4864, hit rate 0.954; MCP-style transcript cacheRead=4608, hit rate 0.891.

Why the assertions differ: Anthropic exposes explicit cache breakpoints and moving conversation-history reuse, while OpenAI's effective reusable prefix in live traffic can plateau earlier than the full prompt. Comparing the two providers against a single cross-provider percentage threshold produces false regressions.

diagnostics.cacheTrace config

diagnostics:
  cacheTrace:
    enabled: true
    filePath: "~/.openclaw/logs/cache-trace.jsonl" # optional
    includeMessages: false # default true
    includePrompt: false # default true
    includeSystem: false # default true

Defaults:

Env toggles (one-off debugging)

What to inspect

• Cache trace events appear as JSONL entries containing staged snapshots such as session:loaded, prompt:before, stream:context, and session:after.
• The per-turn effect of cached tokens shows up in standard usage interfaces: cacheRead and cacheWrite are visible in /usage tokens, /status, session usage summaries, and custom messages.usageTemplate layouts.
• For Anthropic, both cacheRead and cacheWrite should appear when caching is enabled.
• For OpenAI, cacheRead is expected on cache hits; cacheWrite is filled only in Responses API payloads that explicitly include it (refer to OpenAI above).
• OpenAI also returns tracing and rate-limit headers, including x-request-id, openai-processing-ms, and x-ratelimit-*. Use these for request tracing, but rely on the usage payload rather than headers for cache-hit accounting.

Quick troubleshooting

• High cacheWrite on most turns: examine system-prompt inputs for volatility; confirm the model and provider support your cache configuration.
• High cacheWrite on Anthropic: typically indicates the cache breakpoint lands on content that changes with every request.
• Low OpenAI cacheRead: ensure the stable prefix is positioned at the start, the repeated prefix is at least 1024 tokens, and the same prompt_cache_key is reused across turns that should share a cache.
• No effect from cacheRetention: verify the model key matches agents.defaults.models["provider/model"].
• Bedrock Nova requests with cache settings: expected behavior, as these resolve to no cache retention at runtime.

Related docs:

• Anthropic
• Token use and costs
• Session pruning
• Gateway configuration reference

Related

• Token use and costs
• API usage and costs