Local Model Services: On-Demand Server Launch for OpenClaw

models.providers.<id>.localService launches a local model server owned by the provider on demand. When a model or embedding request targets that provider, OpenClaw checks the health endpoint, launches the process if it is not running, waits until it is ready, and then delivers the request. This approach avoids keeping expensive local servers active throughout the day.

How it works

A model or embedding request is routed to a provider that has been configured.
When the provider includes localService, OpenClaw performs a probe against healthUrl.
If the probe succeeds, OpenClaw relies on the server that is already active.
If the probe fails, OpenClaw initiates command using args.
OpenClaw keeps polling the health endpoint until readyTimeoutMs runs out.
The request proceeds through the standard model or embedding transport.
When OpenClaw launched the process and idleStopMs is configured, it terminates the process once the last in-flight request has remained idle for that duration.

OpenClaw does not set up launchd, systemd, Docker, or any other daemon for this purpose. The server operates as a straightforward child process of whichever OpenClaw process triggered it first.

Startup is serialized for each configured provider and its command, argument, and environment set, so simultaneous chat and embedding requests for the same service never create duplicate servers. Each request holds its own lease until response handling finishes, meaning idle shutdown waits for every in-flight model and embedding request. Configured provider aliases stay separate: two aliases can reference different GPU hosts without merging into the same Ollama, LM Studio, or OpenAI-compatible adapter id.

If another OpenClaw process already maintains a healthy server at the same healthUrl, this process uses it without taking ownership (each process manages only the child it started itself). Startup and exit logs contain bounded, redacted child-output tails along with timing and exit details; configured environment values are never exposed.

Managed llama.cpp

The official llama.cpp provider generates this shape automatically. Its guided setup installs a pinned, verified llama-server, writes an absolute command and router preset, picks a free loopback port, and saves the resulting baseUrl and localService config. Chat and local embeddings lease the same managed router through the standard OpenAI-compatible transports.

Do not copy a generated command path across machines. Run llama.cpp setup on each Gateway host so OpenClaw selects and verifies the matching platform build. See llama.cpp Provider.

Config shape

{
  models: {
    providers: {
      local: {
        baseUrl: "http://127.0.0.1:8000/v1",
        apiKey: "local-model",
        api: "openai-completions",
        timeoutSeconds: 300,
        localService: {
          command: "/absolute/path/to/server",
          args: ["--host", "127.0.0.1", "--port", "8000"],
          cwd: "/absolute/path/to/working-dir",
          env: { LOCAL_MODEL_CACHE: "/absolute/path/to/cache" },
          healthUrl: "http://127.0.0.1:8000/v1/models",
          readyTimeoutMs: 180000,
          idleStopMs: 0,
        },
        models: [
          {
            id: "my-local-model",
            name: "My Local Model",
            reasoning: false,
            input: ["text"],
            cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
            contextWindow: 131072,
            maxTokens: 8192,
          },
        ],
      },
    },
  },
}

Put timeoutSeconds on the provider entry (not localService) so slow cold starts and long generations do not trigger the default model request timeout. Specify an explicit healthUrl whenever your server exposes readiness somewhere other than /models on the base URL.

Fields

Field	Required	Description
`command`	yes	Absolute executable path. No shell PATH lookup.
`args`	no	Process arguments. No shell expansion, pipes, globbing, or quoting.
`cwd`	no	Working directory for the process.
`env`	no	Environment variables merged over the OpenClaw process environment.
`healthUrl`	no	Readiness URL. Defaults to `baseUrl` with `/models` appended (`http://127.0.0.1:8000/v1` becomes `http://127.0.0.1:8000/v1/models`).
`readyTimeoutMs`	no	Startup readiness deadline. Default: `120000`.
`idleStopMs`	no	Idle shutdown delay for an OpenClaw-started process. `0` or omitted keeps it alive until OpenClaw exits.

Inferrs example

Inferrs acts as a custom OpenAI-compatible /v1 backend, so the same localService API works with an inferrs provider entry:

{
  agents: {
    defaults: {
      model: { primary: "inferrs/google/gemma-4-E2B-it" },
    },
  },
  models: {
    mode: "merge",
    providers: {
      inferrs: {
        baseUrl: "http://127.0.0.1:8080/v1",
        apiKey: "inferrs-local",
        api: "openai-completions",
        timeoutSeconds: 300,
        localService: {
          command: "/opt/homebrew/bin/inferrs",
          args: [
            "serve",
            "google/gemma-4-E2B-it",
            "--host",
            "127.0.0.1",
            "--port",
            "8080",
            "--device",
            "metal",
          ],
          healthUrl: "http://127.0.0.1:8080/v1/models",
          readyTimeoutMs: 180000,
          idleStopMs: 0,
        },
        models: [
          {
            id: "google/gemma-4-E2B-it",
            name: "Gemma 4 E2B (inferrs)",
            reasoning: false,
            input: ["text"],
            cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
            contextWindow: 131072,
            maxTokens: 4096,
            compat: { requiresStringContent: true },
          },
        ],
      },
    },
  },
}

Swap command with the output of which inferrs on the machine running OpenClaw. Full inferrs setup: Inferrs.

ds4 example

{
  models: {
    providers: {
      ds4: {
        baseUrl: "http://127.0.0.1:18000/v1",
        apiKey: "ds4-local",
        api: "openai-completions",
        timeoutSeconds: 300,
        localService: {
          command: "<DS4_DIR>/ds4-server",
          args: [
            "--model",
            "<DS4_DIR>/ds4flash.gguf",
            "--host",
            "127.0.0.1",
            "--port",
            "18000",
            "--ctx",
            "32768",
            "--tokens",
            "128",
          ],
          cwd: "<DS4_DIR>",
          healthUrl: "http://127.0.0.1:18000/v1/models",
          readyTimeoutMs: 300000,
          idleStopMs: 0,
        },
        models: [],
      },
    },
  },
}

Full setup, context sizing, and verification commands: ds4.

Local models, Local model setup, provider choices, and safety guidance.
Inferrs, Run OpenClaw through the inferrs OpenAI-compatible local server.