Deepgram Speech-to-Text for Voice Notes and Calls

Deepgram provides speech-to-text capabilities. OpenClaw relies on it for transcribing incoming audio and voice-note messages via tools.media.audio, as well as for streaming speech recognition during Voice Call sessions through plugins.entries.voice-call.config.streaming.

When handling batch transcription, the full audio file gets sent to Deepgram, and the resulting transcript is fed into the reply pipeline using the {{Transcript}} and [Audio] blocks. For Voice Call streaming, live G.711 u-law frames are forwarded to Deepgram's WebSocket listen endpoint, with partial and final transcripts emitted as they arrive from Deepgram.

Getting started

Set your API key

DEEPGRAM_API_KEY=dg_...

Enable the audio provider

{
  tools: {
    media: {
      models: [{ provider: "deepgram", model: "nova-3", capabilities: ["audio"] }],
      audio: {
        enabled: true,
      },
    },
  },
}

Send a voice note

Transcription kicks in automatically when an audio message arrives on any connected channel. OpenClaw hands the audio to Deepgram, then inserts the returned transcript into the reply pipeline.

Configuration options

providerOptions.deepgram lets you append extra query parameters straight onto the Deepgram /listen request, so any parameter Deepgram supports can be used (including detect_language, punctuate, smart_format):

With language hint

{
  tools: {
    media: {
      models: [
        { provider: "deepgram", model: "nova-3", language: "en", capabilities: ["audio"] },
      ],
      audio: {
        enabled: true,
      },
    },
  },
}

With Deepgram options

{
  tools: {
    media: {
      models: [{ provider: "deepgram", model: "nova-3", capabilities: ["audio"] }],
      audio: {
        enabled: true,
        providerOptions: {
          deepgram: {
            detect_language: true,
            punctuate: true,
            smart_format: true,
          },
        },
      },
    },
  },
}

Voice Call streaming STT

The included deepgram plugin additionally registers a realtime transcription provider that the Voice Call plugin can use.

{
  plugins: {
    entries: {
      "voice-call": {
        config: {
          streaming: {
            enabled: true,
            provider: "deepgram",
            providers: {
              deepgram: {
                apiKey: "${DEEPGRAM_API_KEY}",
                model: "nova-3",
                endpointingMs: 800,
                language: "en-US",
              },
            },
          },
        },
      },
    },
  },
}

To point at a Deepgram custom endpoint, assign baseUrl the endpoint root, including any base path but omitting /listen. Supported schemes for realtime endpoints are http://, https://, ws://, and wss://. HTTP gets converted to WS, HTTPS to WSS, while explicit WebSocket schemes remain as provided. Session setup fails when the URL is malformed or uses an unsupported scheme.

Note

Telephony audio from Voice Call arrives as 8 kHz G.711 u-law. The Deepgram streaming provider is configured with encoding: "mulaw" and sampleRate: 8000 by default, which means Twilio media frames can be passed through without conversion.

Notes

Authentication

The standard provider auth order is followed for authentication. Going with DEEPGRAM_API_KEY is the easiest approach.

Proxy and custom endpoints

When operating behind a proxy, you can override endpoints or headers on the Deepgram tools.media.models[] entry.

Output behavior

Output adheres to the same audio constraints as the rest of the providers, including limits on file size, timeout durations, and how transcripts are inserted.

Related

• Media tools, A summary of the processing chain for audio, images, and video.

• Configuration, Complete configuration guide, covering media tool options as well.

• Troubleshooting, Typical problems and their fixes, plus debugging guidance.

• FAQ, Answers to common questions regarding OpenClaw setup.