Local Models for OpenClaw

Local models function, but they demand more from hardware, context windows, and defenses against prompt injection: smaller or heavily quantized versions cut context short and bypass safety filters that providers normally apply. This section covers advanced local configurations and custom servers that speak the OpenAI protocol. For the easiest starting point, try LM Studio or Ollama with openclaw onboard.

For servers that should launch only when a chosen model is needed, refer to Local model services.

Hardware floor

Plan for 2+ fully loaded Mac Studios or a comparable GPU setup (roughly $30k+) to get a smooth agent experience. A single 24 GB GPU manages only simpler prompts, and with higher latency. Always deploy the biggest / unquantized version your hardware supports, since smaller or aggressively quantized models increase prompt-injection exposure (check Security).

Pick a backend

Backend	Use when
ds4	Local DeepSeek V4 Flash on macOS Metal with OpenAI-compatible tool calls
LM Studio	First-time local setup, GUI loader, native Responses API
LiteLLM / OAI-proxy / custom OpenAI-compatible proxy	You front another model API and need OpenClaw to treat it as OpenAI
MLX / vLLM / SGLang	High-throughput self-hosted serving with an OpenAI-compatible HTTP endpoint
Ollama	CLI workflow, model library, hands-off systemd service

Go with api: "openai-responses" when the backend supports it (LM Studio does). If not, fall back to api: "openai-completions". When api is left out on a custom provider that uses a baseUrl, OpenClaw assumes openai-completions.

Warning

WSL2 + Ollama + NVIDIA/CUDA: the official Ollama Linux installer turns on a systemd service with Restart=always. On WSL2 GPU setups, autostart may reload the last model during boot and lock host memory, which can trigger repeated VM restarts. See WSL2 crash loop.

LM Studio + large local model (Responses API)

This setup is the strongest local option available right now. In LM Studio, load a large model (a full-size Qwen, DeepSeek, or Llama build), turn on the local server (default http://127.0.0.1:1234), and rely on the Responses API so reasoning stays separate from the final output.

{
  agents: {
    defaults: {
      model: { primary: "lmstudio/my-local-model" },
      models: {
        "anthropic/claude-opus-4-6": { alias: "Opus" },
        "lmstudio/my-local-model": { alias: "Local" },
      },
    },
  },
  models: {
    mode: "merge",
    providers: {
      lmstudio: {
        baseUrl: "http://127.0.0.1:1234/v1",
        apiKey: "lmstudio",
        api: "openai-responses",
        models: [
          {
            id: "my-local-model",
            name: "Local Model",
            reasoning: false,
            input: ["text"],
            cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
            contextWindow: 196608,
            maxTokens: 8192,
          },
        ],
      },
    },
  },
}

Setup checklist:

Get LM Studio from https://lmstudio.ai
Fetch the largest model build on offer (skip "small" or heavily quantized versions), launch the server, and verify http://127.0.0.1:1234/v1/models shows it.
Swap my-local-model for the actual model ID that LM Studio displays.
Leave the model resident; cold-loading adds startup delay.
Tweak contextWindow/maxTokens if your LM Studio version differs.
On WhatsApp, use the Responses API exclusively so only final text goes out.
Retain models.mode: "merge" so hosted models remain as fallback options.

Hybrid config: hosted primary, local fallback

{
  agents: {
    defaults: {
      model: {
        primary: "anthropic/claude-sonnet-4-6",
        fallbacks: ["lmstudio/my-local-model", "anthropic/claude-opus-4-6"],
      },
      models: {
        "anthropic/claude-sonnet-4-6": { alias: "Sonnet" },
        "lmstudio/my-local-model": { alias: "Local" },
        "anthropic/claude-opus-4-6": { alias: "Opus" },
      },
    },
  },
  models: {
    mode: "merge",
    providers: {
      lmstudio: {
        baseUrl: "http://127.0.0.1:1234/v1",
        apiKey: "lmstudio",
        api: "openai-responses",
        models: [
          {
            id: "my-local-model",
            name: "Local Model",
            reasoning: false,
            input: ["text"],
            cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
            contextWindow: 196608,
            maxTokens: 8192,
          },
        ],
      },
    },
  },
}

For local-first operation with a hosted backup, reverse the primary/fallbacks order and preserve the same providers block plus models.mode: "merge".

Regional hosting / data routing

Hosted MiniMax/Kimi/GLM variants are also available on OpenRouter with region-locked endpoints (US-hosted, for instance). Choose the regional variant to keep traffic within your preferred jurisdiction while keeping models.mode: "merge" for Anthropic/OpenAI fallbacks. Local-only remains the most private path; hosted regional routing sits in between when you need provider features but want control over data movement.

Other OpenAI-compatible local proxies

MLX (mlx_lm.server), vLLM, SGLang, LiteLLM, OAI-proxy, or any custom gateway works as long as it exposes an OpenAI-style /v1/chat/completions endpoint. Use openai-completions unless the backend explicitly states /v1/responses support.

{
  agents: {
    defaults: {
      model: { primary: "local/my-local-model" },
    },
  },
  models: {
    mode: "merge",
    providers: {
      local: {
        baseUrl: "http://127.0.0.1:8000/v1",
        apiKey: "sk-local",
        api: "openai-completions",
        timeoutSeconds: 300,
        models: [
          {
            id: "my-local-model",
            name: "Local Model",
            reasoning: false,
            input: ["text"],
            cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
            contextWindow: 120000,
            maxTokens: 8192,
          },
        ],
      },
    },
  },
}

Custom and local provider entries trust the exact configured baseUrl origin for guarded model requests, which covers loopback, LAN, tailnet, and private DNS hosts. Metadata, link-local, and local-use NAT64 (64:ff9b:1::/48) origins stay blocked until explicitly enabled. Requests aimed at other private origins still require models.providers.<id>.request.allowPrivateNetwork: true; set the trust flag to false to skip exact-origin trust.

models.providers.<id>.models[].id is scoped to the provider, so leave off the provider prefix. For an MLX server started with mlx_lm.server --model mlx-community/Qwen3-30B-A3B-6bit:

models.providers.mlx.models[].id: "mlx-community/Qwen3-30B-A3B-6bit"
agents.defaults.model.primary: "mlx/mlx-community/Qwen3-30B-A3B-6bit"

Set input: ["text", "image"] on local or proxied vision models so image attachments are inserted into agent turns. Interactive custom-provider onboarding guesses common vision model IDs and only asks about unfamiliar ones; non-interactive onboarding applies the same guessing, with --custom-image-input / --custom-text-input available to override it.

Use models.providers.<id>.timeoutSeconds for slow local or remote model servers before bumping agents.defaults.timeoutSeconds. The provider timeout applies to connect, headers, body streaming, and the total guarded-fetch abort for model HTTP requests only. If the agent or run timeout is lower, raise that as well, since the provider timeout cannot stretch the whole run.

Note

For custom OpenAI-compatible providers, a non-secret local marker such as apiKey: "ollama-local" is accepted when baseUrl resolves to loopback, a private LAN, .local, or a bare hostname. OpenClaw treats it as a valid local credential instead of flagging a missing key. Use a real value for any provider that accepts a public hostname.

Behavior notes for local or proxied /v1 backends:

OpenClaw treats these as proxy-style OpenAI-compatible routes, not native OpenAI endpoints.
Native-OpenAI-only request shaping does not apply: no service_tier, no Responses store, no OpenAI reasoning-compat payload shaping, no prompt-cache hints.
Hidden OpenClaw attribution headers (originator, version, User-Agent) are not injected on custom proxy URLs.

Compat declarations are only for the custom endpoint described by this provider row. Catalog-known routes use provider-owned capabilities instead; see the custom-provider capability guide.

Compat overrides for stricter OpenAI-compatible backends:

String-only content: some servers accept only string messages[].content, not structured content-part arrays. Set models.providers.<provider>.models[].compat.requiresStringContent: true.
Strict message keys: if the server rejects message entries with more than role/content, set compat.strictMessageKeys: true.
Bracketed tool text: some local models emit standalone bracketed tool requests as text, like [tool_name] followed by JSON and [END_TOOL_REQUEST]. OpenClaw promotes those to real tool calls only when the name exactly matches a registered tool for the turn; otherwise it stays as hidden, unsupported text.
Unstructured tool-call-looking text: if a model emits JSON/XML/ReAct-style text that looks like a tool call but wasn't a structured invocation, OpenClaw leaves it as text and logs a warning with the run id, provider/model, detected pattern, and tool name when available. That is provider/model incompatibility, not a completed tool run.
Forcing tool use: if tools show up as assistant text (raw JSON/XML/ReAct, or an empty tool_calls array), first confirm the server's chat template/parser supports tool calls. If the parser only works when tool use is forced, override the default proxy value of tool_choice: "auto" per model:
```
{
  agents: {
    defaults: {
      models: {
        "local/my-local-model": {
          params: {
            extra_body: {
              tool_choice: "required",
            },
          },
        },
      },
    },
  },
}
```
Use this only where every normal turn should call a tool. Replace local/my-local-model with the exact ref from openclaw models list, or set it via CLI:
```
openclaw config set agents.defaults.models '{"local/my-local-model":{"params":{"extra_body":{"tool_choice":"required"}}}}' --strict-json --merge
```

Extra reasoning efforts: if a custom OpenAI-compatible model accepts OpenAI reasoning efforts beyond the built-in profile, declare them in the model's compat block. Adding "xhigh" exposes it for that model ref in /think xhigh, session pickers, Gateway validation, and llm-task validation:

{
  models: {
    providers: {
      local: {
        baseUrl: "http://127.0.0.1:8000/v1",
        apiKey: "sk-local",
        api: "openai-responses",
        models: [
          {
            id: "gpt-5.4",
            name: "GPT 5.4 via local proxy",
            reasoning: true,
            input: ["text"],
            cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
            contextWindow: 196608,
            maxTokens: 8192,
            compat: {
              supportedReasoningEfforts: ["low", "medium", "high", "xhigh"],
              reasoningEffortMap: { xhigh: "xhigh" },
            },
          },
        ],
      },
    },
  },
}

Smaller or stricter backends

If the model loads cleanly but full agent turns misbehave, work top-down: confirm transport first, then narrow the surface.

Confirm the local model responds - no tools, no agent context:

openclaw infer model run --local --model <provider/model> --prompt "Reply with exactly: pong" --json

Confirm Gateway routing - sends only the prompt, skipping transcript, AGENTS bootstrap, context-engine assembly, tools, and bundled MCP servers, but still exercises Gateway routing, auth, and provider selection:
```
openclaw infer model run --gateway --model <provider/model> --prompt "Reply with exactly: pong" --json
```
Try lean mode if both probes pass but real agent turns fail with malformed tool calls or oversized prompts: set agents.defaults.experimental.localModelLean: true. It drops heavyweight browser, cron, message, media-generation, voice, and PDF tools unless explicitly required, and defaults larger tool catalogs behind structured Tool Search controls while keeping exec directly visible. See Experimental Features -> Local model lean mode for details and how to confirm it's on.
Disable tools entirely as a last resort by setting models.providers.<provider>.models[].compat.supportsTools: false for that model - the agent then runs without tool calls.
Past that, the bottleneck is upstream. If the backend still fails only on larger OpenClaw runs after lean mode and supportsTools: false, the remaining issue is usually the model or server itself - context window, GPU memory, kv-cache eviction, or a backend bug - not OpenClaw's transport layer.

Troubleshooting

Gateway can't reach the proxy? curl http://127.0.0.1:1234/v1/models.
LM Studio model unloaded? Reload; cold start is a common "hanging" cause.
Local server says terminated, ECONNRESET, or closes the stream mid-turn? OpenClaw records a low-cardinality model.call.error.failureKind plus the OpenClaw process RSS/heap snapshot in diagnostics. For LM Studio/Ollama memory pressure, match that timestamp against the server log or a macOS crash/jetsam log to confirm whether the model server was killed.
Context errors? OpenClaw derives context-window preflight thresholds from the detected model window or the per-model models.providers.<provider>.models[].contextTokens cap, warning below 20% with an 8k floor and hard-blocking below 10% with a 4k floor. Lower that model entry's contextTokens or raise the server/model context limit.
messages[].content ... expected a string? Add compat.requiresStringContent: true on that model entry.
validation.keys, or "message entries only allow role and content"? Add compat.strictMessageKeys: true on that model entry.
Direct /v1/chat/completions calls work, but openclaw infer model run --local fails on Gemma or another local model? Check the provider URL, model ref, auth marker, and server logs first - model run skips agent tools entirely. If model run succeeds but larger agent turns fail, reduce the tool surface with localModelLean or compat.supportsTools: false.
Tool calls show up as raw JSON/XML/ReAct text, or the provider returns an empty tool_calls array? Do not add a proxy that blindly converts assistant text into tool execution - fix the server's chat template/parser first. If the model only works when tool use is forced, add the params.extra_body.tool_choice: "required" override above and use that model entry only for sessions where a tool call is expected every turn.
Safety: local models skip provider-side filters. Keep agents narrow and compaction on to limit prompt-injection blast radius.