Deepgram Speech-to-Text for Audio and Voice Call

Deepgram provides speech-to-text as an API. Within OpenClaw it handles transcription of inbound audio and voice notes via tools.media.audio, and it supplies streaming STT for Voice Call through plugins.entries.voice-call.config.streaming.

With batch transcription, the entire audio file is uploaded to Deepgram. Flux models take a different route and rely on Deepgram's one-shot WebSocket. Either way, the resulting transcript is fed into the reply pipeline ({{Transcript}} + [Audio] block). For Voice Call streaming, live G.711 u-law frames are forwarded to Deepgram's WebSocket listen endpoint, and partial and final transcripts are emitted as soon as Deepgram returns them.

Getting started

Set your API key

export DEEPGRAM_API_KEY=dg_...

Enable the audio provider

{
  tools: {
    media: {
      models: [{ provider: "deepgram", model: "nova-3", capabilities: ["audio"] }],
      audio: {
        enabled: true,
      },
    },
  },
}

Send a voice note

Send an audio message over any connected channel. Deepgram transcribes it, and OpenClaw injects the transcript into the reply pipeline.

Configuration options

Any extra query params passed to providerOptions.deepgram are merged straight into the Deepgram /listen request, so every param name Deepgram supports is accepted (detect_language, punctuate, and smart_format are examples).

For array values, repeated query parameters are sent in both batch and Flux requests. As an example, set providerOptions on the audio config or on a model entry so that each keyterm goes out separately rather than as a comma-separated list:

{
  "providerOptions": {
    "deepgram": {
      "keyterm": ["OpenClaw", "Deepgram"]
    }
  }
}

With language hint

{
  tools: {
    media: {
      models: [
        { provider: "deepgram", model: "nova-3", language: "en", capabilities: ["audio"] },
      ],
      audio: {
        enabled: true,
      },
    },
  },
}

With Deepgram options

{
  tools: {
    media: {
      models: [{ provider: "deepgram", model: "nova-3", capabilities: ["audio"] }],
      audio: {
        enabled: true,
        providerOptions: {
          deepgram: {
            detect_language: true,
            punctuate: true,
            smart_format: true,
          },
        },
      },
    },
  },
}

Flux models

For Deepgram Flux, use flux-general-en or flux-general-multi. OpenClaw first converts the voice note to 16 kHz mono linear16 audio using ffmpeg, then uploads it to Deepgram's /v2/listen WebSocket endpoint. In the getting-started configuration shown above, set tools.media.models[].model to one of the Flux models.

Voice-note conversion works on the whole audio file, including notes that run longer than 20 minutes. The decoded audio is held in a private temporary file that gets deleted once the transcription attempt finishes. The input-size and request-timeout limits you configured still apply. If conversion or upload fails, no partial transcript is returned.

In providerOptions.deepgram, Flux accepts eager_eot_threshold, eot_threshold, eot_timeout_ms, keyterm, language_hint, mip_opt_out, numerals, profanity_filter, redact, and tag. Batch-only options like detect_language, punctuate, and smart_format are ignored by OpenClaw on Flux. Language hints work only with flux-general-multi: the model entry's language setting is mapped by OpenClaw to language_hint, and an explicit provider option wins over it. For the English-only flux-general-en model, both settings have no effect.

Voice Call streaming STT

A realtime transcription provider for the Voice Call plugin is also registered by the bundled deepgram plugin.

{
  plugins: {
    entries: {
      "voice-call": {
        config: {
          streaming: {
            enabled: true,
            provider: "deepgram",
            providers: {
              deepgram: {
                apiKey: "${DEEPGRAM_API_KEY}",
                model: "nova-3",
                endpointingMs: 800,
                language: "en-US",
              },
            },
          },
        },
      },
    },
  },
}

When you use a Deepgram custom endpoint, point baseUrl at the endpoint root. That root should carry any base path, but it must not include /listen. The realtime endpoints accept http://, https://, ws://, and wss://. HTTP becomes WS and HTTPS becomes WSS, while an explicit WebSocket scheme is left as it is. If a URL is malformed or uses another scheme, session setup fails.

Note

Telephony audio reaches Voice Call as 8 kHz G.711 u-law. Because the Deepgram streaming provider ships with encoding: "mulaw" and sampleRate: 8000 as its defaults, Twilio media frames can be passed straight through.

Notes

Authentication

Auth works through the usual provider auth order. The easiest route is DEEPGRAM_API_KEY.

Proxy and custom endpoints

If you are behind a proxy, override the endpoints or headers on the Deepgram tools.media.models[] entry.

Output behavior

Output obeys the same audio rules that apply to every other provider (size caps, timeouts, transcript injection).

Flux requires ffmpeg

Before you pick a Flux model, install ffmpeg using the package manager on the gateway host.

Related

• Media tools, Overview of the audio, image, and video processing pipeline.

• Configuration, Complete config reference, media tool settings included.

• Troubleshooting, Typical problems and how to debug them.

• FAQ, Common questions about setting up OpenClaw.