Inferrs Provider: Run OpenClaw with Local OpenAI-Compatible Models

inferrs runs local models through an API that mirrors OpenAI's /v1 interface. OpenClaw connects to it via the standard openai-completions adapter.

Note

inferrs is a custom self-hosted backend compatible with OpenAI's format, not a dedicated OpenClaw provider plugin: you configure it under models.providers.inferrs rather than selecting an authentication option during onboarding. For a bundled plugin with automatic detection, check SGLang or vLLM.

Getting started

Start inferrs with a model

inferrs serve google/gemma-4-E2B-it \
  --host 127.0.0.1 \
  --port 8080 \
  --device metal

Verify the server is reachable

curl http://127.0.0.1:8080/health
curl http://127.0.0.1:8080/v1/models

Add an OpenClaw provider entry

Create an explicit provider entry and direct your default model to it. The configuration example below shows how.

Full config example

Gemma 4 running on a local inferrs server:

{
  agents: {
    defaults: {
      model: { primary: "inferrs/google/gemma-4-E2B-it" },
      models: {
        "inferrs/google/gemma-4-E2B-it": {
          alias: "Gemma 4 (inferrs)",
        },
      },
    },
  },
  models: {
    mode: "merge",
    providers: {
      inferrs: {
        baseUrl: "http://127.0.0.1:8080/v1",
        apiKey: "inferrs-local",
        api: "openai-completions",
        models: [
          {
            id: "google/gemma-4-E2B-it",
            name: "Gemma 4 E2B (inferrs)",
            reasoning: false,
            input: ["text"],
            cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
            contextWindow: 131072,
            maxTokens: 4096,
            compat: {
              requiresStringContent: true,
            },
          },
        ],
      },
    },
  },
}

On-demand startup

OpenClaw can launch inferrs on its own only when an inferrs/... model is chosen. Add localService to the same provider entry:

{
  models: {
    providers: {
      inferrs: {
        baseUrl: "http://127.0.0.1:8080/v1",
        apiKey: "inferrs-local",
        api: "openai-completions",
        timeoutSeconds: 300,
        localService: {
          command: "/opt/homebrew/bin/inferrs",
          args: [
            "serve",
            "google/gemma-4-E2B-it",
            "--host",
            "127.0.0.1",
            "--port",
            "8080",
            "--device",
            "metal",
          ],
          healthUrl: "http://127.0.0.1:8080/v1/models",
          readyTimeoutMs: 180000,
          idleStopMs: 0,
        },
        models: [
          {
            id: "google/gemma-4-E2B-it",
            name: "Gemma 4 E2B (inferrs)",
            reasoning: false,
            input: ["text"],
            cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
            contextWindow: 131072,
            maxTokens: 4096,
            compat: {
              requiresStringContent: true,
            },
          },
        ],
      },
    },
  },
}

command must be a full absolute path. Execute which inferrs on the Gateway host and use the resulting path. For a complete field reference, see Local model services.

Advanced configuration

Why requiresStringContent matters

Certain inferrs Chat Completions endpoints only accept string messages[].content values, not structured arrays of content parts.

Warning

If OpenClaw runs fail with:

messages[1].content: invalid type: sequence, expected a string

set compat.requiresStringContent: true in the model entry. OpenClaw then converts plain text content parts into simple strings before sending the request.

Gemma and tool-schema caveat

Some inferrs and Gemma combinations handle small direct /v1/chat/completions requests fine but break on full OpenClaw agent runtime turns. First try turning off the tool schema surface:

compat: {
  requiresStringContent: true,
  supportsTools: false
}

That lowers prompt pressure on stricter local backends. If tiny direct requests still succeed but normal OpenClaw agent turns keep failing inside inferrs, consider it an upstream model or server limitation rather than an OpenClaw transport problem.

Manual smoke test

Once configured, test both layers:

curl http://127.0.0.1:8080/v1/chat/completions \
  -H 'content-type: application/json' \
  -d '{"model":"google/gemma-4-E2B-it","messages":[{"role":"user","content":"What is 2 + 2?"}],"stream":false}'

openclaw infer model run \
  --model inferrs/google/gemma-4-E2B-it \
  --prompt "What is 2 + 2? Reply with one short sentence." \
  --json

If the first command works but the second does not, refer to Troubleshooting below.

Proxy-style behavior

Because inferrs uses the generic openai-completions adapter (not openai-responses), native-OpenAI-only request shaping never applies: no service_tier, no Responses store, no prompt-cache hints, and no OpenAI reasoning-compat payload shaping get sent.

Troubleshooting

curl /v1/models fails

inferrs is not running, not reachable, or not bound to the host/port you configured. Confirm the server is started and listening on that address.

messages[].content expected a string

Set compat.requiresStringContent: true in the model entry (see above).

Direct /v1/chat/completions calls pass but openclaw infer model run fails

Set compat.supportsTools: false to disable the tool schema surface (see the Gemma caveat above).

inferrs still crashes on larger agent turns

If schema errors are gone but inferrs still crashes on larger agent turns, treat it as an upstream inferrs or model limitation. Reduce prompt pressure or switch backend/model.

Tip

For general help, see Troubleshooting and FAQ.

Related

• Local models, Using OpenClaw with model servers hosted on your own machine.
• Local model services, Launching local model servers on demand for specified providers.
• Gateway troubleshooting, Diagnosing local OpenAI-compatible backends that pass health checks but cause agent execution failures.
• Model selection, Summary of every provider, model reference, and failover logic.