ComfyUI Workflow-Based Image, Video, and Music Generation in OpenClaw

Install the official comfy plugin to run ComfyUI through workflow-based execution:

openclaw plugins install @openclaw/comfy-provider
openclaw gateway restart

Everything the plugin does is driven by workflows. OpenClaw never translates standard size, aspectRatio, resolution, durationSeconds, or TTS-style commands into operations on your graph.

What it supports

• Generate or edit images from a workflow JSON. Editing accepts a single uploaded reference image.
• Produce video from a workflow JSON, either text-to-video or image-to-video with one reference image.
• Create music or audio via the shared music_generate tool, optionally passing one reference image.
• Fetch output from a designated node, or from every matching output node when none is specified.

Getting started

You can run ComfyUI locally on your own hardware, or rely on Comfy Cloud instead.

Local

Best for: operating your own ComfyUI instance on your machine or LAN.

Start ComfyUI locally

Confirm your local ComfyUI instance is active, which by default listens on http://127.0.0.1:8188.

Prepare your workflow JSON

Export or put together a ComfyUI workflow JSON file. Keep track of the node IDs for the prompt input node and the output node that OpenClaw should pull results from.

Configure the provider

Configure mode: "local" to reference your workflow file. A minimal image example:

{
  plugins: {
    entries: {
      comfy: {
        config: {
          mode: "local",
          baseUrl: "http://127.0.0.1:8188",
          image: {
            workflowPath: "./workflows/flux-api.json",
            promptNodeId: "6",
            outputNodeId: "9",
          },
        },
      },
    },
  },
}

Set the default model

For the capability you set up, direct OpenClaw to the comfy/workflow model:

{
  agents: {
    defaults: {
      mediaModels: {
        image: {
          primary: "comfy/workflow",
        },
      },
    },
  },
}

Verify

openclaw models list --provider comfy

Comfy Cloud

Best for: executing workflows on Comfy Cloud without handling local GPU resources.

Get an API key

Register at comfy.org and create an API key from your account dashboard.

Set the API key

Supply your key using any of the following approaches:

# Onboarding flag
openclaw onboard --comfy-api-key "your-key"

# Environment variable (preferred for daemons)
export COMFY_API_KEY="your-key"

# Alternative environment variable
export COMFY_CLOUD_API_KEY="your-key"

# Or inline in config
openclaw config set plugins.entries.comfy.config.apiKey "your-key"

Prepare your workflow JSON

Export or create a ComfyUI workflow JSON file. Record the node IDs for the prompt input node and the output node.

Configure the provider

Set mode: "cloud" and reference your workflow file:

{
  plugins: {
    entries: {
      comfy: {
        config: {
          mode: "cloud",
          image: {
            workflowPath: "./workflows/flux-api.json",
            promptNodeId: "6",
            outputNodeId: "9",
          },
        },
      },
    },
  },
}

Tip

Cloud mode sets baseUrl to https://cloud.comfy.org by default. Only assign baseUrl when you need a custom cloud endpoint.

Set the default model

{
  agents: {
    defaults: {
      mediaModels: {
        image: {
          primary: "comfy/workflow",
        },
      },
    },
  },
}

Verify

openclaw models list --provider comfy

Configuration

Comfy supports shared top-level connection settings alongside per-capability workflow sections (image, video, music):

{
  plugins: {
    entries: {
      comfy: {
        config: {
          mode: "local",
          baseUrl: "http://127.0.0.1:8188",
          image: {
            workflowPath: "./workflows/flux-api.json",
            promptNodeId: "6",
            outputNodeId: "9",
          },
          video: {
            workflowPath: "./workflows/video-api.json",
            promptNodeId: "12",
            outputNodeId: "21",
          },
          music: {
            workflowPath: "./workflows/music-api.json",
            promptNodeId: "3",
            outputNodeId: "18",
          },
        },
      },
    },
  },
}

Shared keys

When your ComfyUI instance sits behind HTTP authentication, go with headers.Authorization. Credentials are best handled via a secret reference. Headers are sent with uploads, workflow submissions, polling, and downloads, regardless of the mode. They take precedence over default headers in a case-insensitive manner, with one exception: Content-Type is not overridden on image uploads, since the runtime sets the multipart boundary. If a header's SecretRef is unavailable, the request fails before being sent. Any reflected header values are stripped from response errors.

Note

In local mode, loopback/private IP literals and single-label service names like http://comfyui:8188 function without allowPrivateNetwork. Public-looking private-DNS FQDNs such as https://comfy.local.example.com demand allowPrivateNetwork: true. Trust for private origins stays limited to the configured scheme, hostname, and port; local redirects are confined to the configured hostname, while cloud redirects to public CDNs undergo checks with the default SSRF policy.

Per-capability keys

These keys are valid within the image, video, or music sections:

The image and video sections additionally support a reference-image input node:

apiKey takes either a literal string or a secret reference object.

Workflow details

Image workflows

Configure the default image model as comfy/workflow:

{
  agents: {
    defaults: {
      mediaModels: {
        image: {
          primary: "comfy/workflow",
        },
      },
    },
  },
}

Reference-image editing example:

To turn on image editing with an uploaded reference image, include inputImageNodeId in your image config:

{
  plugins: {
    entries: {
      comfy: {
        config: {
          image: {
            workflowPath: "./workflows/edit-api.json",
            promptNodeId: "6",
            inputImageNodeId: "7",
            inputImageInputName: "image",
            outputNodeId: "9",
          },
        },
      },
    },
  },
}

Video workflows

Set the default video model to comfy/workflow:

{
  agents: {
    defaults: {
      mediaModels: {
        video: {
          primary: "comfy/workflow",
        },
      },
    },
  },
}

Through the configured graph, Comfy video workflows handle both text-to-video and image-to-video.

Note

OpenClaw never sends input videos into Comfy workflows. Only text prompts and a single reference image are accepted as inputs.

Music workflows

The bundled plugin registers a music-generation provider for audio or music outputs defined in workflows, exposed through the shared music_generate tool. It optionally accepts one reference image (max 1):

/tool music_generate prompt="Warm ambient synth loop with soft tape texture"

Point to your audio workflow JSON and output node using the music config section.

Backward compatibility

Top-level image config that predates the nested image section remains functional:

{
  plugins: {
    entries: {
      comfy: {
        config: {
          workflowPath: "./workflows/flux-api.json",
          promptNodeId: "6",
          outputNodeId: "9",
        },
      },
    },
  },
}

OpenClaw recognizes this older layout as the configuration for image workflows. While migration can wait, the structured image / video / music blocks are suggested for fresh installations. When image generation is your sole concern, the older flat arrangement and the newer nested image block behave the same.

Live tests

Live testing for the included plugin is available on an opt-in basis:

OPENCLAW_LIVE_TEST=1 COMFY_LIVE_TEST=1 pnpm test:live -- extensions/comfy/comfy.live.test.ts

Unless the corresponding Comfy workflow block is set up, the live test skips any individual image, video, or music test case.

Related

• Image Generation, Configuration and usage details for the image generation tool.

• Video Generation, Configuration and usage details for the video generation tool.

• Music Generation, Setup for the music and audio generation tool.

• Provider Directory, A summary of every provider and model reference.

• Configuration reference, The complete configuration reference, including agent defaults.