Image and Media Support

The WhatsApp channel is built on Baileys Web. This document explains how media is handled for sends, gateway operations, and agent responses.

For details on inline audio and video within the Control UI and native applications, covering portable formats, byte restrictions, and lazy transcoding, refer to Media playback.

Goals

Attach media with an optional caption using openclaw message send --media.
Enable the web inbox to send automatic replies that combine media with text.
Establish per-type limits that are reasonable and consistent.

CLI Surface

openclaw message send --target <dest> --media <path-or-url> [--message <caption>]

--media <path-or-url>, add media (image/audio/video/document); local paths or URLs are accepted. Optional; for media-only sends, the caption may be left empty.
--gif-playback, render video media as GIF playback (WhatsApp only).
--force-document, keep original image bytes intact on Slack, or transmit images, GIFs, and videos as documents on Telegram and WhatsApp to prevent channel compression.
--reply-to <id>, --thread-id <id>, --pin, --silent, delivery and threading options identical to those for text-only sends.
--dry-run, output the resolved payload and refrain from sending.
--json, output the result as JSON: { action, channel, dryRun, handledBy, messageId?, payload } (payload holds the channel-specific send outcome, including any media reference).

WhatsApp Web channel behavior

Input: a local file path or an HTTP(S) URL.
Process: read into a buffer, identify the media type, then construct the outbound payload according to type:
- Images: optimized to stay under channels.whatsapp.mediaMaxMb (default 50MB). Opaque images are recompressed to JPEG (default side ladder begins at 2048px, decreasing on repeated size failures); images with transparency remain as PNG. If the source is already a suitable JPEG/PNG/WebP within the size and side-length limits, the original bytes are kept unchanged rather than recompressed. Animated GIFs are never re-encoded, only checked for size.
- Audio/voice: unless already native voice audio (.ogg/.opus, or audio/ogg/audio/opus), outbound audio is transcoded via ffmpeg to Opus/OGG (48kHz mono, 64kbps, capped at 20 minutes) before being sent as a voice note (ptt: true).
- Video: pass-through up to 16MB.
- Documents: all other content, up to 100MB, with filename preserved when available.
WhatsApp GIF-style playback: send an MP4 with gifPlayback: true (CLI: --gif-playback) so mobile clients loop it inline.
MIME detection prefers sniffed magic bytes, then the file extension, then response headers; a generic sniffed container (application/octet-stream, zip) never overrides a more specific extension mapping (for example XLSX vs ZIP).
Caption comes from --message or reply.text; an empty caption is permitted.
Logging: non-verbose shows ↩️/✅; verbose includes size and source path/URL.

Note

The 16MB audio/video and 100MB document figures above are the shared per-kind media defaults used when no explicit byte cap is passed. WhatsApp sends set an explicit cap from channels.whatsapp.mediaMaxMb (default 50MB), which applies uniformly across kinds for that account.

Auto-Reply Pipeline

getReplyFromConfig returns a reply payload (or array of payloads) with text?, mediaUrl?, and mediaUrls? among other fields.
When media is present, the web sender resolves local paths or URLs using the same pipeline as openclaw message send.
Multiple media entries are sent sequentially if provided.

Inbound Media To Commands

When inbound web messages include media, OpenClaw downloads it to a temp file and exposes templating variables:
- {{AttachmentUrl}}, original URL or provider reference for the current attachment.
- {{AttachmentPath}}, local temp path written before running the command.
- {{AttachmentContentType}}, MIME content type.
- {{AttachmentDir}}, directory containing the local path.
- {{AttachmentIndex}}, zero-based source fact index.
When a per-session Docker sandbox is enabled, inbound media is copied into the sandbox workspace and the attachment path/reference is rewritten to a sandbox-relative path like media/inbound/<filename>.
{{MediaPath}}, {{MediaUrl}}, {{MediaType}}, and {{MediaDir}} remain deprecated compatibility aliases during the plugin SDK migration window.
Media understanding (configured via tools.media.* or shared tools.media.models) runs before templating and can insert [Image], [Audio], and [Video] blocks into Body.
- Audio sets {{Transcript}} and uses the transcript for command parsing so slash commands still work.
- Video and image descriptions preserve any caption text for command parsing.
- If the active primary model already supports vision natively, OpenClaw skips the [Image] summary block and passes the original image to the model instead.
By default only the first matching image/audio/video attachment is processed; use tools.media.<capability>.attachments to select multiple attachments.

Limits and errors

Outbound send caps (WhatsApp web send)

After optimization, images can be up to channels.whatsapp.mediaMaxMb (50MB by default).
Audio and video are capped at 16MB (shared default; mediaMaxMb overrides this when sending via WhatsApp).
Documents are capped at 100MB (shared default; mediaMaxMb overrides this when sending via WhatsApp).
If media is too large or unreadable, a clear error appears in the logs and the reply is skipped.

Media understanding caps (transcription/description)

Images default to 10MB (adjustable with tools.media.image.maxBytes, or per tools.media.models[] entry using maxBytes).
Audio defaults to 20MB (adjustable with tools.media.audio.maxBytes, or per entry).
Video defaults to 50MB (adjustable with tools.media.video.maxBytes, or per entry).
Oversized media skips understanding, but the reply still goes through with the original body.

Notes for Tests

Cover send and reply flows for image/audio/document cases.
Validate size bounds after image optimization and the voice-note flag for audio.
Ensure multi-media replies fan out as sequential sends.