Prompt Caching: Knobs, Merge Order, and Provider Behavior

Prompt caching allows a model provider to reuse a prefix that has not changed (system or developer instructions, tool definitions, and other stable context) across multiple turns, rather than reprocessing it with each request. The result is lower token consumption and reduced latency for long-lived sessions that repeatedly send the same context.

Where the upstream API exposes cache counters, OpenClaw normalizes provider usage into cacheRead and cacheWrite. When the live session snapshot lacks cache counters, usage summaries (such as /status) fall back to the last transcript usage entry; a nonzero live value takes precedence over the fallback.

Provider references:

• Anthropic prompt caching
• OpenAI prompt caching

Primary knobs

cacheRetention

Values: "none" | "short" | "long". This can be configured as a global default, per model, or per agent. "standard" is not an alias; for the provider's default cache window, use "short". Invalid values trigger a warning and are ignored.

agents:
  defaults:
    params:
      cacheRetention: "long" # none | short | long
    models:
      "anthropic/claude-opus-4-6":
        params:
          cacheRetention: "short" # overrides the global default for this model
  list:
    - id: "alerts"
      params:
        cacheRetention: "none" # overrides both defaults for this agent

Merge order (later wins):

1. agents.defaults.params - global default for all models
2. agents.defaults.models["provider/model"].params - per-model override
3. agents.entries.*.params - per-agent override, matched by agent id

Source: src/agents/embedded-agent-runner/extra-params.ts (resolveExtraParams).

contextPruning.mode: "cache-ttl"

Once the cache TTL window expires, old tool-result context is pruned, so a post-idle request does not re-cache oversized history.

agents:
  defaults:
    contextPruning:
      mode: "cache-ttl"
      ttl: "1h"

Full behavior is described in Session pruning.

Heartbeat keep-warm

A heartbeat can keep cache windows warm and cut down on repeated cache writes after idle gaps. It is configurable globally (agents.defaults.heartbeat) or per agent (agents.entries.*.heartbeat).

agents:
  defaults:
    heartbeat:
      every: "55m"

Provider behavior

Anthropic (direct API and Vertex AI)

• For anthropic and anthropic-vertex providers, and for Claude models on amazon-bedrock and custom anthropic-messages-compatible endpoints when cacheRetention is set explicitly, cacheRetention is supported.
• When unset, OpenClaw seeds cacheRetention: "short" for direct Anthropic (anthropic and anthropic-vertex providers only; other Anthropic-family routes need an explicit value).
• From native Anthropic Messages responses, cache_read_input_tokens and cache_creation_input_tokens are exposed and mapped to cacheRead and cacheWrite.
• The default 5-minute ephemeral cache maps to cacheRetention: "short". An explicit cacheRetention: "long" requests the 1-hour TTL (cache_control: { type: "ephemeral", ttl: "1h" }). An implicit/env-driven long retention (OPENCLAW_CACHE_RETENTION=long with no explicit cacheRetention) only upgrades to the 1-hour TTL on api.anthropic.com or Vertex AI (aiplatform.googleapis.com / *-aiplatform.googleapis.com) hosts; other hosts retain the 5-minute cache.

Source: packages/ai/src/transports/anthropic-payload-policy.ts (resolveAnthropicEphemeralCacheControl, isLongTtlEligibleEndpoint).

OpenAI (direct API)

• For recent supported models, prompt caching happens automatically; OpenClaw never adds cache markers at the block level.
• To keep cache routing stable across turns, OpenClaw sends prompt_cache_key. Direct api.openai.com hosts receive this automatically. For OpenAI-compatible proxies such as oMLX, llama.cpp, or custom endpoints, you must add compat.supportsPromptCacheKey: true to the model config to enable it; OpenClaw never detects this automatically for a proxy.
• prompt_cache_retention: "24h" appears only when cacheRetention: "long" is chosen and the resolved endpoint supports both the cache key and long retention (compat.supportsLongCacheRetention, which defaults to true; Together AI and Cloudflare compat profiles turn it off). cacheRetention: "none" suppresses both fields.
• Cache hits show up via usage.prompt_tokens_details.cached_tokens (Chat Completions) or input_tokens_details.cached_tokens (Responses API), which map to cacheRead.
• Responses API payloads may also expose input_tokens_details.cache_write_tokens, mapped to cacheWrite and billed at the model's cache-write rate; Responses payloads without that field keep cacheWrite at 0. OpenAI's Chat Completions API neither documents nor emits a cache_write_tokens counter, but OpenClaw still reads prompt_tokens_details.cache_write_tokens there for OpenRouter-compatible and DeepSeek-style proxies that report a separate write count.
• In practice, OpenAI behaves more like an initial-prefix cache than Anthropic's moving full-history reuse, as described in OpenAI live expectations below.

Amazon Bedrock

• Anthropic Claude model refs (amazon-bedrock/*anthropic.claude*, plus AWS system inference profile prefixes us./eu./global.anthropic.claude*) support explicit cacheRetention pass-through.
• Non-Anthropic Bedrock models (for example amazon.nova-*) resolve to no cache retention at runtime, regardless of any configured cacheRetention value.
• Opaque Bedrock application inference profile ARNs (profile IDs that do not contain claude) also resolve to no cache retention unless cacheRetention is set explicitly, since the model family cannot be inferred from the ARN alone.

OpenRouter

For openrouter/anthropic/* model refs, OpenClaw injects Anthropic cache_control markers on system/developer prompt blocks, but only when the request still targets a verified OpenRouter route (openrouter on its default endpoint, or any provider/base URL that resolves to openrouter.ai). Repointing the model at an arbitrary OpenAI-compatible proxy URL stops this injection.

contextPruning.mode: "cache-ttl" is allowed for openrouter/anthropic/*, openrouter/deepseek/*, openrouter/moonshot/*, openrouter/moonshotai/*, and openrouter/zai/* model refs, because these routes handle provider-side prompt caching without needing OpenClaw's injected markers.

Source: extensions/openrouter/index.ts (OPENROUTER_CACHE_TTL_MODEL_PREFIXES).

DeepSeek cache construction on OpenRouter is best-effort and can take a few seconds; an immediate follow-up request may still show cached_tokens: 0. Verify with a repeated same-prefix request after a short delay, using usage.prompt_tokens_details.cached_tokens as the cache-hit signal.

Google Gemini (direct API)

• Direct Gemini transport (api: "google-generative-ai") reports cache hits through upstream cachedContentTokenCount, mapped to cacheRead.
• Eligible model families: gemini-2.5* and gemini-3* (excludes Live/preview variants outside that prefix match, for example gemini-live-2.5-flash-preview).
• When cacheRetention is set on an eligible model, OpenClaw automatically creates, reuses, and refreshes a cachedContents resource for the system prompt, with no manual cached-content handle needed. TTL is 300s for cacheRetention: "short" and 3600s for "long".
• You can still pass a pre-existing Gemini cached-content handle through as params.cachedContent (or legacy params.cached_content); an explicit handle skips the automatic cache-management path entirely.
• This is separate from Anthropic/OpenAI prompt-prefix caching: OpenClaw manages a provider-native cachedContents resource for Gemini instead of injecting inline cache markers.

Source: src/agents/embedded-agent-runner/google-prompt-cache.ts.

CLI-harness providers (Claude Code, Gemini CLI)

CLI backends that produce JSONL usage events (jsonlDialect: "claude-stream-json" or "gemini-stream-json") share a common usage parser, which accepts several field-name variants, among them a bare cached counter that maps to cacheRead. When the CLI's JSON payload lacks a direct input-token field, OpenClaw computes it as input_tokens - cached. This is purely usage normalization; it does not generate Anthropic/OpenAI-style prompt-cache markers for these CLI-driven models.

Source: src/agents/cli-output.ts (toCliUsage).

Other providers

When a provider supports none of the cache modes listed above, cacheRetention does nothing.

System-prompt cache boundary

OpenClaw divides the system prompt into a stable prefix and a volatile suffix at an internal cache-prefix boundary. Content above that boundary (tool definitions, skills metadata, workspace files) is arranged to remain byte-identical from turn to turn. Content below it (such as runtime timestamps and other per-turn metadata) may shift without breaking the cached prefix.

Key design choices:

• Stable workspace project-context files come before volatile per-turn metadata, so routine churn does not disrupt the stable prefix.
• The boundary applies across Anthropic-family, OpenAI-family, Google, and CLI transport shaping, giving every supported provider the same prefix stability.
• Codex Responses and Anthropic Vertex requests go through boundary-aware cache shaping, so cache reuse lines up with what providers actually receive.
• System-prompt fingerprints are normalized (whitespace, line endings, hook-added context, runtime capability ordering), so semantically unchanged prompts share cache across turns.

If you notice unexpected cacheWrite spikes after a config or workspace change, check whether the change falls above or below the cache boundary. Moving volatile content below the boundary, or stabilizing it, typically fixes the problem.

OpenClaw cache-stability guards

• Bundled MCP tool catalogs are sorted deterministically (by server name, then tool name) before tool registration, so listTools() order changes do not churn the tools block and break prompt-cache prefixes.
• Legacy sessions with persisted image blocks keep the 3 most recent completed turns intact (counting all completed turns, not just image-bearing ones). Older already-processed image blocks are replaced with a text marker, so image-heavy follow-ups do not keep re-sending large stale payloads.

Tuning patterns

Mixed traffic (recommended default)

Maintain a long-lived baseline on your main agent, and disable caching on bursty notifier agents:

agents:
  defaults:
    model:
      primary: "anthropic/claude-opus-4-6"
    models:
      "anthropic/claude-opus-4-6":
        params:
          cacheRetention: "long"
  list:
    - id: "research"
      default: true
      heartbeat:
        every: "55m"
    - id: "alerts"
      params:
        cacheRetention: "none"

Cost-first baseline

• Set baseline cacheRetention: "short".
• Enable contextPruning.mode: "cache-ttl".
• Keep heartbeat below your TTL only for agents that benefit from warm caches.

Live regression tests

OpenClaw runs one combined live cache regression gate covering repeated prefixes, tool turns, image turns, MCP-style tool transcripts, and an Anthropic no-cache control.

• src/agents/live-cache-regression.live.test.ts
• src/agents/test-helpers/live-cache-regression-runner.ts
• src/agents/live-cache-regression-baseline.ts

Run it with:

OPENCLAW_LIVE_TEST=1 OPENCLAW_LIVE_CACHE_TEST=1 pnpm test:live:cache

The baseline file stores the most recently observed live numbers plus the provider-specific regression floors the test checks against. Each run uses fresh per-run session IDs and prompt namespaces so previous cache state does not pollute the current sample. Anthropic and OpenAI use different enforcement: an Anthropic floor miss is a hard regression (test fails), while an OpenAI floor miss is watch-only (recorded as a warning, does not fail the run). They do not share a single cross-provider threshold.

Anthropic live expectations

• Expect explicit warmup writes via cacheWrite.
• Expect near-full history reuse on repeated turns, because Anthropic's cache control advances the cache breakpoint through the conversation.
• Baseline floors for stable, tool, image, and MCP-style lanes are hard regression gates.

OpenAI live expectations

• Expect cacheRead only; cacheWrite stays 0 on Chat Completions.
• Treat repeated-turn cache reuse as a provider-specific plateau, not Anthropic-style moving full-history reuse.
• Floors are watch-only (a miss is logged as a warning, not a test failure), derived from observed live behavior on gpt-5.4-mini:

The most recently observed baseline numbers (from live-cache-regression-baseline.ts) landed at: stable prefix cacheRead=4864, hit rate 0.966; tool transcript cacheRead=4608, hit rate 0.896; image transcript cacheRead=4864, hit rate 0.954; MCP-style transcript cacheRead=4608, hit rate 0.891.

Why the assertions differ: Anthropic exposes explicit cache breakpoints and moving conversation-history reuse, while OpenAI's effective reusable prefix in live traffic can plateau earlier than the full prompt. Comparing the two providers against a single cross-provider percentage threshold produces false regressions.

diagnostics.cacheTrace config

diagnostics:
  cacheTrace:
    enabled: true

enabled defaults to false. Cache traces otherwise write to $OPENCLAW_STATE_DIR/logs/cache-trace.jsonl and include messages, prompt text, and the system prompt by default. Output-path and payload-inclusion overrides are environment-only controls for one-off debugging.

Env toggles (one-off debugging)

What to inspect

Cache trace events use JSONL format and capture staged snapshots, for example session:loaded, prompt:before, stream:context, and session:after.

The effect of per-turn cache tokens appears in standard usage views: cacheRead and cacheWrite are reflected in /usage tokens, /status, session usage summaries, and custom messages.usageTemplate layouts.

With Anthropic, both cacheRead and cacheWrite should appear while caching is enabled.

For OpenAI, cacheRead appears on cache hits; cacheWrite is only set when the Responses API payload includes it (refer to OpenAI above).

OpenAI additionally returns tracing and rate-limit headers like x-request-id, openai-processing-ms, and x-ratelimit-*; rely on these for request tracing, but derive cache-hit counts from the usage payload rather than the headers.

Quick troubleshooting

• Elevated cacheWrite on most turns: inspect for volatile system-prompt inputs; confirm the model and provider support your cache settings.
• Elevated cacheWrite on Anthropic: this usually indicates the cache breakpoint falls on content that changes with each request.
• Low OpenAI cacheRead: ensure the stable prefix appears first, the repeated prefix is at least 1024 tokens, and the same prompt_cache_key is used across turns meant to share a cache.
• cacheRetention has no effect: check that the model key matches agents.defaults.models["provider/model"].
• Bedrock Nova requests with cache settings: this is normal, they resolve to no cache retention at runtime.

Related documentation:

• Anthropic
• Token use and costs
• Session pruning
• Gateway configuration reference

Related

• Token use and costs
• API usage and costs