Neura Market Logo

Adding a New Shared Capability to the OpenClaw Plugin System

This guide explains when and how to add a new shared capability to the OpenClaw plugin system. It is intended for core contributors who need to define a vendor-agnostic contract for features like embeddings or image generation.

Read this when

  • Adding a new core capability and plugin registration surface
  • Deciding whether code belongs in core, a vendor plugin, or a feature plugin
  • Wiring a new runtime helper for channels or tools

Info

This guide is intended for contributors working on the OpenClaw core. If you are developing an external plugin, refer to Building plugins instead. For a detailed architectural overview covering the capability model, ownership, load pipeline, and runtime utilities, see Plugin internals.

Follow this approach when OpenClaw requires a new shared domain, such as embeddings, image generation, video generation, or another future vendor-supported feature area.

The fundamental principle:

  • A plugin defines an ownership boundary.
  • A capability defines a shared core contract.

Never connect a vendor directly to a channel or tool. Always define the capability first.

When to create a capability

Create a new capability only when all of the following conditions hold:

  1. Multiple vendors could realistically implement it.
  2. Channels, tools, or feature plugins should be able to use it without knowing which vendor is behind it.
  3. The core must manage fallback logic, policies, configuration, or delivery behavior.

If the work involves only a single vendor and no shared contract exists yet, define that contract before proceeding.

The standard sequence

  1. Define the typed core contract.
  2. Add plugin registration for that contract.
  3. Add a shared runtime helper.
  4. Wire one real vendor plugin as a proof of concept.
  5. Move feature and channel consumers to the runtime helper.
  6. Add contract tests.
  7. Document the operator-facing configuration and ownership model.

What goes where

LayerOwns
CoreRequest and response types; provider registry and resolution; fallback behavior; config schema with propagated title/description docs metadata on nested objects, wildcards, array items, and composition nodes; runtime helper interface.
Vendor pluginVendor API calls, vendor authentication handling, vendor-specific request normalization, and registration of the capability implementation.
Feature/channel pluginCalls api.runtime.* or the corresponding plugin-sdk/*-runtime helper. Never calls a vendor implementation directly.

Provider and harness seams

Use provider hooks when the behavior belongs to the model provider contract rather than the generic agent loop. Examples include provider-specific request parameters after transport selection, auth profile preference, prompt overlays, and follow-up fallback routing after model or profile failover.

Use agent harness hooks when the behavior belongs to the runtime executing a turn. Harnesses can classify explicit protocol outcomes, such as empty output, reasoning without visible output, or a structured plan without a final answer, so the outer model fallback policy can decide whether to retry.

Keep both seams narrow:

  • Core owns the retry and fallback policy.
  • Provider plugins own provider-specific request, auth, and routing hints.
  • Harness plugins own runtime-specific attempt classification.
  • Third-party plugins return hints, not direct mutations of core state.

File checklist

For a new capability, expect to modify these areas:

  • src/<capability>/types.ts
  • src/<capability>/...registry/runtime.ts
  • src/plugins/types.ts
  • src/plugins/registry.ts
  • src/plugins/captured-registration.ts
  • src/plugins/contracts/registry.ts
  • src/plugins/runtime/types-core.ts
  • src/plugins/runtime/index.ts
  • src/plugin-sdk/<capability>.ts
  • src/plugin-sdk/<capability>-runtime.ts
  • One or more bundled plugin packages.
  • Configuration, documentation, and tests.

Worked example: image generation

Image generation follows the standard pattern:

  1. Core defines ImageGenerationProvider.
  2. Core exposes registerImageGenerationProvider(...).
  3. Core exposes api.runtime.imageGeneration.generate(...) and .listProviders(...).
  4. Vendor plugins (comfy, deepinfra, fal, google, litellm, microsoft-foundry, minimax, openai, openrouter, vydra, xai) register vendor-backed implementations.
  5. Future vendors register the same contract without requiring changes to channels or tools.

The config key is intentionally separate from vision analysis routing:

  • agents.defaults.imageModel is used for analyzing images.
  • agents.defaults.mediaModels.image is used for generating images.

Keep these separate so that fallback logic and policy remain explicit.

Embedding providers

Use registerEmbeddingProvider(...) and contract embeddingProviders for reusable vector embedding providers. This contract is intentionally broader than memory: tools, search, retrieval, importers, or future feature plugins can consume embeddings without depending on the memory engine. Memory search also consumes generic embeddingProviders.

The older memory-specific registration API and memoryEmbeddingProviders contract are deprecated. Use registerEmbeddingProvider and embeddingProviders for all new embedding providers.

Review checklist

Before shipping a new capability, verify the following:

  • No channel or tool imports vendor code directly.
  • The runtime helper is the shared path.
  • At least one contract test asserts bundled ownership.
  • Config documentation names the new model or config key.
  • Plugin documentation explains the ownership boundary.

If a pull request bypasses the capability layer and hardcodes vendor behavior into a channel or tool, send it back and define the contract first.