Voice Mode in Hermes Agent: CLI, Telegram, Discord Voice Channels

{ "title": "Hermes Agent Voice Mode Reference", "body": "## Overview\n\nVoice mode enables full-duplex spoken interaction with Hermes Agent. You can speak into a microphone and hear spoken replies, send voice messages in Telegram or Discord and receive spoken audio responses, or have the agent join a Discord voice channel to listen and speak to multiple users. The system supports streaming text-to-speech (TTS), barge-in (interrupting the agent mid-speech), wake word activation, configurable silence detection, and a hallucination filter that suppresses phantom transcriptions from background noise.\n\n## Prerequisites\n\nBefore enabling voice features, ensure the following are in place:\n\n- Hermes Agent installed via the install script.\n- An LLM provider configured (run hermes model or set credentials in ~/.hermes/.env).\n- A working base setup: run hermes to verify the agent responds to text before enabling voice.\n- Python packages installed for the desired voice features (voice, messaging, tts-premium, or all).\n- System dependencies installed (PortAudio, ffmpeg, Opus, espeak-ng as needed).\n- API keys configured in ~/.hermes/.env for cloud STT/TTS providers (optional for local).\n- faster-whisper installed for local STT (optional, no key needed).\n- Discord bot set up with appropriate permissions and intents for Discord features.\n- Opus codec installed on the server for Discord voice channels.\n- Telegram bot set up for Telegram features.\n- ffmpeg installed for audio format conversion.\n- For Docker: host audio socket access configured.\n- For NeuTTS: espeak-ng installed and the neutts package installed separately.\n- For Nous Portal: a paid subscription and run hermes setup --portal on a fresh install.\n- ~/.hermes/.env created manually for API keys (config.yaml is auto-created on the first hermes run).\n\n## Installation\n\n### Install Voice Dependencies\n\nInstall the Python packages for voice features using uv pip. All commands assume you are in the ~/.hermes/hermes-agent directory.\n\nTo install CLI voice mode packages:\n\n\ncd ~/.hermes/hermes-agent && uv pip install -e \".[voice]\"\n\n\nTo install Discord and Telegram messaging packages:\n\n\ncd ~/.hermes/hermes-agent && uv pip install -e \".[messaging]\"\n\n\nTo install premium TTS (ElevenLabs):\n\n\ncd ~/.hermes/hermes-agent && uv pip install -e \".[tts-premium]\"\n\n\nTo install local TTS (NeuTTS, optional):\n\n\npython -m pip install -U neutts[all]\n\n\nTo install all packages at once:\n\n\ncd ~/.hermes/hermes-agent && uv pip install -e \".[all]\"\n\n\n### Install System Dependencies\n\nSystem dependencies vary by operating system.\n\nmacOS:\n\n\nbrew install portaudio ffmpeg opus\n\n\nFor NeuTTS on macOS:\n\n\nbrew install espeak-ng\n\n\nUbuntu/Debian:\n\n\nsudo apt install portaudio19-dev ffmpeg libopus0\n\n\nFor NeuTTS on Ubuntu/Debian:\n\n\nsudo apt install espeak-ng\n\n\n### Configure API Keys\n\nAPI keys are stored in ~/.hermes/.env. Create this file manually with the following entries as needed:\n\n- For local STT (faster-whisper): no key needed, install faster-whisper.\n- For Groq Whisper: set GROQ_API_KEY=your-key.\n- For OpenAI Whisper/TTS: set VOICE_TOOLS_OPENAI_KEY=your-key.\n- For ElevenLabs TTS: set ELEVENLABS_API_KEY=***.\n- Edge TTS and NeuTTS work without any key.\n\n## CLI Voice Mode\n\n### Starting and Using CLI Voice Mode\n\n1. Start the CLI: hermes.\n2. Enable voice mode: /voice on.\n3. Press Ctrl+B to start recording. A beep at 880 Hz plays.\n4. Speak into the microphone. A live audio level bar shows input.\n5. Stop speaking. After 3 seconds of silence, recording auto-stops. Two beeps at 660 Hz play.\n6. Audio is transcribed and sent to the agent.\n7. If TTS is enabled, the agent's reply is spoken aloud.\n8. Recording automatically restarts for the next input.\n9. To exit continuous mode, press Ctrl+B during recording.\n10. Voice chat ends after three consecutive silent cycles or by saying a stop phrase.\n\n### CLI Voice Commands\n\n- /voice – Toggle voice mode.\n- /voice on – Enable voice mode.\n- /voice off – Disable voice mode.\n- /voice tts – Enable voice replies for all messages.\n- /voice status – Show current voice mode setting.\n\n### Silence Detection in CLI\n\nSpeech is confirmed when audio stays above the RMS threshold (default 200) for at least 0.3 seconds. The end of speech is detected after 3.0 seconds of continuous silence. Both silence_threshold and silence_duration are configurable in config.yaml.\n\n### Recording Behavior\n\n- Recording auto-restarts after a reply unless stopped by Ctrl+B or three silent cycles.\n- If no speech is detected for 15 seconds, recording stops automatically.\n- The record key is configurable via voice.record_key in config.yaml.\n- Beep sounds can be disabled via voice.beep_enabled: false.\n\n## Gateway Voice Reply (Telegram and Discord)\n\nThe gateway provides voice reply functionality for Telegram and Discord messaging platforms.\n\n### Setup\n\n1. Set up messaging bots per platform-specific guides.\n2. Start the gateway: hermes gateway.\n3. Or run the interactive setup wizard: hermes gateway setup.\n\n### Commands\n\n- /voice on – Enable voice replies when you send a voice message.\n- /voice tts – Enable voice replies for all messages.\n- /voice off – Disable voice replies.\n- /voice status – Show current setting.\n\n### Voice Mode States\n\nVoice mode has three states:\n\n- off – Text only.\n- voice_only – Reply only to voice messages.\n- all – Reply to all messages.\n\n### Telegram Voice Bubbles\n\nTelegram voice bubble format is Opus/OGG. ffmpeg converts MP3 to Opus if needed.\n\n### Discord Voice Bubbles\n\nDiscord voice bubble format is Opus/OGG. Falls back to file attachment if the API fails.\n\n## Discord Voice Channels\n\n### Setup\n\n1. Go to Discord Developer Portal, select your application, then Installation, then Default Install Settings, then Guild Install.\n2. Add Connect, Speak, and Use Voice Activity permissions to existing text permissions.\n3. Use the updated permissions integer: 309240908864.\n4. Re-invite the bot with this URL: https://discord.com/oauth2/authorize?client_id=YOUR_APP_ID&scope=bot+applications.commands&permissions=309240908864.\n5. In Developer Portal, under Bot, Privileged Gateway Intents, enable Presence Intent, Server Members Intent (if using usernames in DISCORD_ALLOWED_USERS), and Message Content Intent.\n6. Install the Opus codec on the server: macOS: brew install opus; Ubuntu/Debian: sudo apt install libopus0.\n7. Set environment variables in ~/.hermes/.env: DISCORD_BOT_TOKEN, DISCORD_ALLOWED_USERS, and optionally STT/TTS keys.\n8. Start the gateway: hermes gateway.\n\n### Using Discord Voice Channels\n\n1. Ensure you are in a voice channel.\n2. In a text channel where the bot is present, run: /voice join.\n3. The bot joins your current voice channel.\n4. Speak. The bot listens, detects silence (1.5 seconds after 0.5 seconds of speech), transcribes, processes, and speaks a reply.\n5. Transcripts appear in the text channel: [Voice] @user: what you said.\n6. Agent responses are sent as text in the channel and spoken in the voice channel.\n7. To disconnect: /voice leave.\n8. To check status: /voice status.\n\n### Discord Voice Channel Commands\n\n- /voice join – Bot joins your current voice channel.\n- /voice leave – Bot leaves the voice channel.\n- /voice status – Show current voice channel status.\n\n### Access Control\n\nOnly users listed in DISCORD_ALLOWED_USERS can interact via voice in the voice channel. Others are silently ignored. DISCORD_ALLOWED_USERS can be numeric IDs or usernames. Usernames require the Server Members Intent.\n\n### Text Channel Integration\n\nTranscripts and agent responses appear in the text channel where /voice join was issued.\n\n### Echo Prevention\n\nThe bot pauses its audio listener while playing TTS replies to prevent echo.\n\n### Multiple Users\n\nThe bot supports multiple users independently in the same voice channel.\n\n### Silence Detection in Discord Voice Channels\n\nSilence detection uses 1.5 seconds of silence after at least 0.5 seconds of speech to trigger processing.\n\n## Streaming TTS\n\nThe agent speaks sentence-by-sentence as it generates text. This works with all TTS providers.\n\n- Providers with chunked PCM API (ElevenLabs, OpenAI) stream raw audio for lowest latency.\n- Other providers synthesize and play each sentence as it completes.\n- The same TTS pipeline runs in classic CLI, TUI, and desktop app.\n- In desktop voice conversation, reply text is fed live into a per-reply speech WebSocket as the model generates.\n- One socket and one audio clock per reply, no per-sentence connection gaps.\n\n### TTS Pipeline Details\n\n- Buffers text into sentences (minimum 20 characters) before speaking.\n- Strips markdown, emoji, and <think> blocks before speaking.\n\n## Barge-In\n\nBarge-in allows you to interrupt the agent mid-generation or mid-playback by speaking, typing, pressing the record key, or saying a stop phrase.\n\n- Barge-in during LLM generation interrupts the in-flight turn. The interjection becomes the next message.\n- Barge-in during playback cuts playback and submits what was said.\n- Typing or pressing the record key stops playback instantly.\n- Saying a stop phrase mid-generation interrupts the turn and ends the voice chat. Mid-playback, it cuts speech and ends the chat.\n- The agent knows it was interrupted and can react naturally.\n\n### Barge-In Configuration\n\n- voice.barge_in – Enable/disable barge-in feature (default: true).\n- voice.barge_in_threshold_multiplier – Scales the speech trigger over the quiet-room floor (default: 3.0).\n- voice.barge_in_grace_seconds – Suppresses false trips right after playback starts (default: 0.5).\n\n### Barge-In Grace Period\n\nThe grace period (default 0.5 seconds) prevents false trips right after playback starts. Adjust this value if speaker bleed causes false barge-in.\n\n### Noise Floor Calibration\n\nThe noise floor is calibrated against the quiet room at turn start, not against playback.\n\n## Wake Word Support\n\nSay "hey hermes" (or a custom phrase) to start a fresh voice session hands-free on CLI, TUI, or desktop app.\n\n## Hallucination Filter\n\nThe hallucination filter filters out phantom Whisper transcriptions from silence or background noise. It uses 26 known phrases and a regex pattern for repetitive variations. If issues persist, use a quieter environment or adjust silence_threshold.\n\n## Stop Phrases\n\nStop phrases are configurable. Set the list to [] to disable voice end-by-phrase.\n\n- The stop phrase must be the entire utterance (case-insensitive, surrounding punctuation ignored) to trigger.\n- The match is strict: "stop doing that" will not trigger; it will be sent to the agent normally.\n- Typing a stop phrase while voice chat is active ends the chat instead of sending to the agent.\n- Outside a voice chat, a typed stop phrase is an ordinary message.\n- Voice chat also ends on its own after three consecutive silent cycles.\n\n## Configuration Parameters\n\nAll parameters are set in ~/.hermes/config.yaml unless noted as environment variables.\n\n### Voice Parameters\n\n- voice.record_key – Key binding to start/stop recording in CLI voice mode (default: ctrl+b).\n- voice.max_recording_seconds – Maximum recording length in seconds (default: 120).\n- voice.auto_tts – Auto-enable TTS when voice mode starts (default: false).\n- voice.beep_enabled – Play record start/stop beeps (default: true).\n- voice.silence_threshold – RMS level (0-32767) below which counts as silence (default: 200).\n- voice.silence_duration – Seconds of silence before auto-stop (default: 3.0).\n- voice.stop_phrases – List of phrases that end the voice chat when spoken exactly (default: [\"stop\"]).\n- voice.barge_in – Enable/disable barge-in feature (default: true).\n- voice.barge_in_threshold_multiplier – Scales speech trigger over quiet-room floor (default: 3.0).\n- voice.barge_in_grace_seconds – Suppresses trips right after playback starts (default: 0.5).\n\n### STT Parameters\n\n- stt.enabled – Enable/disable automatic transcription. If false, the audio file path is passed to the agent (default: true).\n- stt.provider – STT provider: local, groq, openai, mistral, xai (default: local).\n- stt.local.model – Local Whisper model size: tiny, base, small, medium, large-v3 (default: base).\n- stt.local.language – Optional ISO-639-1 language hint for local STT. Blank = auto-detect.\n- stt.groq.language – Optional ISO-639-1 language hint for Groq STT. Blank = auto-detect.\n\n### TTS Parameters\n\n- tts.provider – TTS provider: edge, elevenlabs, openai, neutts, minimax, mistral, gemini, xai, kittentts, piper (default: edge).\n- tts.edge.voice – Edge TTS voice name (322 voices, 74 languages) (default: en-US-AriaNeural).\n- tts.elevenlabs.voice_id – ElevenLabs voice ID (default: pNInz6obpgDQGcFmaJgB - Adam).\n- tts.elevenlabs.model_id – ElevenLabs model ID (default: eleven_multilingual_v2).\n- tts.openai.model – OpenAI TTS model (default: gpt-4o-mini-tts).\n- tts.openai.voice – OpenAI TTS voice: alloy, echo, fable, onyx, nova, shimmer (default: alloy).\n- tts.openai.base_url – Override base URL for OpenAI-compatible TTS endpoints.\n- tts.neutts.ref_audio – Reference audio path for NeuTTS.\n- tts.neutts.ref_text – Reference text for NeuTTS.\n- tts.neutts.model – NeuTTS model identifier (default: neuphonic/neutts-air-q4-gguf).\n- tts.neutts.device – Device for NeuTTS (cpu or cuda) (default: cpu).\n\n### Environment Variables\n\n- DISCORD_REQUIRE_MENTION – If false, the bot responds in server channels without @mention.\n- DISCORD_FREE_RESPONSE_CHANNELS – Comma-separated channel IDs where no mention is needed.\n- DISCORD_ALLOWED_USERS – Comma-separated user IDs or usernames allowed to interact via voice in VC. Required for voice channel interaction.\n- DISCORD_BOT_TOKEN – Discord bot token. Required for Discord.\n- GROQ_API_KEY – API key for Groq Whisper STT. Required if using Groq.\n- VOICE_TOOLS_OPENAI_KEY – API key for OpenAI Whisper STT and OpenAI TTS. Required if using OpenAI.\n- ELEVENLABS_API_KEY – API key for ElevenLabs TTS. Required if using ElevenLabs.\n- STT_GROQ_MODEL – Override default Groq STT model.\n- STT_OPENAI_MODEL – Override default OpenAI STT model.\n- GROQ_BASE_URL – Custom Groq API endpoint.\n- STT_OPENAI_BASE_URL – Custom OpenAI STT endpoint.\n- HERMES_VOICE_DEBUG – Set to 1 to stream VAD diagnostics to stderr.\n\n## Provider Details\n\n### STT Providers\n\n- Local (faster-whisper): Free, no key needed. Downloads model (~150 MB for base) on first use. Model options: tiny, base, small, medium, large-v3.\n- Groq: Fast, free tier available. Requires GROQ_API_KEY. Model can be overridden via STT_GROQ_MODEL. Custom endpoint via GROQ_BASE_URL.\n- OpenAI: Paid. Requires VOICE_TOOLS_OPENAI_KEY. Model can be overridden via STT_OPENAI_MODEL. Custom endpoint via STT_OPENAI_BASE_URL.\n- Mistral, xAI: Additional providers.\n\n### TTS Providers\n\n- Edge TTS: Free, 322 voices, 74 languages. Default voice: en-US-AriaNeural. Default free fallback.\n- ElevenLabs: Paid. Requires ELEVENLABS_API_KEY. Configurable voice and model.\n- OpenAI: Paid. Requires VOICE_TOOLS_OPENAI_KEY. Model: gpt-4o-mini-tts. Voices: alloy, echo, fable, onyx, nova, shimmer. Supports voice-design instructions via the text_to_speech tool's instructions argument for tone, emotion, pacing, accent, and whispering. Custom base URL via tts.openai.base_url.\n- NeuTTS: Free, local. Requires separate neutts[all] install. Configurable reference audio, reference text, model, and device.\n- Minimax, Mistral, Gemini, xAI, KittenTTS, Piper: Additional providers.\n\n### Provider Fallback\n\n- STT provider fallback order: local > groq > openai.\n- TTS provider fallback: Edge TTS is the default free fallback if others fail.\n\n## OpenAI TTS Voice Design\n\nFor OpenAI TTS, the text_to_speech tool accepts an optional instructions argument for voice design. This argument controls tone, emotion, pacing, accent, and whispering. It only works with gpt-4o-mini-tts or compatible endpoints. The instructions argument also routes to OpenAI-compatible voice-design servers via tts.openai.base_url.\n\n## Discord Configuration Details\n\n### Permissions\n\nThe Discord permissions integer for text and voice is 309240908864.\n\n### Privileged Gateway Intents\n\n- Presence Intent\n- Server Members Intent (conditional, needed if DISCORD_ALLOWED_USERS uses usernames)\n- Message Content Intent\n\n### Opus Codec\n\nThe bot auto-loads the Opus codec from standard system paths:\n- macOS: /opt/homebrew/lib/libopus.dylib\n- Linux: libopus.so.0\n\n### Mention Requirement\n\n- By default, the bot requires @mention in server channels.\n- Set DISCORD_REQUIRE_MENTION=false to disable globally.\n- Use DISCORD_FREE_RESPONSE_CHANNELS for per-channel exemption.\n- DM mode does not require mention.\n- In server channels, select the bot user from the mention popup, not the role with the same name.\n\n### SSRC to User ID Mapping\n\nThe bot maps SSRC to user_id via Discord's SPEAKING opcode. The Server Members Intent is not needed for voice.\n\n## Voice Mode Persistence\n\nThe voice mode setting is persisted across gateway restarts.\n\n## Logging\n\n- Gateway logs are forwarded to ~/.hermes/logs/gateway.log.\n- TUI forwards crash-forensic logs to ~/.hermes/logs/.\n\n## Debug Mode\n\nSet HERMES_VOICE_DEBUG=1 for per-block VAD diagnostics to stderr.\n\n## Gateway Setup Wizard\n\nRun hermes gateway setup for an interactive setup wizard.\n\n## Nous Portal\n\nA Nous Portal subscription provides LLM and OpenAI TTS via the Tool Gateway. No separate OpenAI key is needed. On a fresh install, hermes setup --portal wires up Nous Portal at once. The cost per use is approximately $0.0045.\n\n## Failure Modes and Troubleshooting\n\n### Audio and System Issues\n\n- No audio device found (CLI): PortAudio is not installed.\n- Docker audio issues: The container needs access to the host audio socket (PulseAudio/PipeWire).\n- Opus codec not found: Install libopus0 (Linux) or opus (macOS).\n- ffmpeg not installed: Audio format conversion fails.\n- NeuTTS model download fails: Check internet connection.\n- faster-whisper model download fails: Check internet connection.\n\n### Discord Issues\n\n- Bot doesn't respond in Discord server channels: @mention is required by default. Use DM or disable mention requirement.\n- Bot joins VC but doesn't hear me: User is not in DISCORD_ALLOWED_USERS, muted in Discord, or no SPEAKING event.\n- Bot hears me but doesn't respond: STT unavailable (install faster-whisper or set API key), LLM not configured, or gateway logs show errors.\n- Bot responds in text but not in voice channel: TTS provider failing (check API key, quota, logs). Edge TTS is the default fallback.\n- Discord voice bubble API fails: Falls back to file attachment.\n- Discord voice channel echo: The bot should pause the listener during playback. Check echo prevention logic.\n- Multiple users in VC: Only DISCORD_ALLOWED_USERS are processed. Others are ignored.\n- Server Members Intent not enabled: Usernames in DISCORD_ALLOWED_USERS will not resolve.\n- Message Content Intent not enabled: The bot cannot read text messages in channels.\n- Bot permissions insufficient: Re-invite with the correct permissions integer.\n- Opus codec not auto-loaded: Check system paths. May need to set LD_LIBRARY_PATH.\n\n### STT and TTS Issues\n\n- Whisper returns garbage text: The hallucination filter catches most cases. Use a quieter environment, adjust silence_threshold, or try a different STT model.\n- API key missing or invalid: STT or TTS provider fails.\n- LLM provider not configured: The agent cannot process messages.\n- TTS not playing: Check TTS provider configuration and API keys.\n- Streaming TTS stuttering: Network latency or provider issues.\n- Hallucination filter missing phrases: Update the list or adjust silence_threshold.\n\n### Configuration and Command Issues\n\n- Gateway fails to start: Check configuration and logs at ~/.hermes/logs/gateway.log.\n- Voice mode setting not persisted: Rare. Check config.yaml permissions.\n- Barge-in not working: Check voice.barge_in setting and threshold multiplier.\n- Stop phrase not working: Check voice.stop_phrases list. The phrase must be an exact match.\n- Recording not stopping: Check silence_threshold and silence_duration settings.\n- Beep sounds not playing: Check voice.beep_enabled setting.\n- Auto-TTS not enabling: Set voice.auto_tts: true.\n- Voice mode not toggling: Check /voice command syntax.\n- Gateway not connecting to platforms: Check network and API keys.\n- Telegram bot not responding: Check bot token and setup.\n- Discord bot not online: Check bot token and gateway status.\n- Voice channel join fails: Ensure the user is in a voice channel.\n- Voice channel leave fails: The bot may not be connected.\n- Voice status shows incorrect state: Check config and gateway logs.\n\n### Provider Fallback Issues\n\n- STT provider fallback not working: Check provider priority order.\n- TTS provider fallback not working: Check provider configuration.\n- Custom base_url not working: Ensure the URL is correct and compatible.\n- OpenAI TTS instructions not applied: Check provider and model compatibility.\n- NeuTTS not working: Check model download and device configuration.\n- ElevenLabs voice not found: Check voice_id.\n- Groq STT model override not applied: Check STT_GROQ_MODEL env var.\n- OpenAI STT model override not applied: Check STT_OPENAI_MODEL env var.\n- Custom Groq endpoint not working: Check GROQ_BASE_URL.\n- Custom OpenAI STT endpoint not working: Check STT_OPENAI_BASE_URL.\n\n### Environment and Setup Issues\n\n- DISCORD_FREE_RESPONSE_CHANNELS not working: Check channel IDs are correct.\n- DISCORD_REQUIRE_MENTION=false not working: Check env var is set correctly.\n- Voice mode setting not persisted across restart: Check config.yaml permissions.\n- Gateway setup wizard fails: Check network and input.\n- Nous Portal setup fails: Check subscription and network.\n- hermes setup --portal fails: Check fresh install and network.\n- ~/.hermes/.env not created: Create manually with required keys.\n- config.yaml not created: Run hermes first to auto-generate.\n- Python package installation fails: Check uv pip and dependencies.\n- System dependency installation fails: Check package manager and permissions.\n- faster-whisper not installed: Install via pip.\n- discord.py[voice] not installed: Install via pip with the messaging extra.\n- PyNaCl not installed: Installed automatically with discord.py[voice].\n- Opus bindings not installed: Installed automatically with discord.py[voice].\n- espeak-ng not installed: Needed for NeuTTS on macOS/Linux.\n- portaudio19-dev not installed: Needed for CLI voice mode on Ubuntu.\n- ffmpeg not installed: Needed for audio format conversion.\n- libopus0 not installed: Needed for Discord voice channels on Ubuntu.\n- opus not installed: Needed for Discord voice channels on macOS.\n\n## Additional Notes\n\n- The ~/.hermes/ directory and default config.yaml are created automatically on the first hermes run.\n- Only ~/.hermes/.env needs manual creation for API keys.\n- If faster-whisper is installed, voice mode works with zero API keys for STT.\n- Model downloads automatically on first use (~150 MB for base).\n- Edge TTS and NeuTTS work without any API key.\n- Re-inviting the bot to a server updates permissions without losing data or configuration.\n- The bot auto-loads the Opus codec from standard system paths.\n- Voice-channel SSRC to user_id mapping comes from Discord's SPEAKING opcode, not Server Members Intent.\n- The TTS pipeline buffers text into sentences (minimum 20 characters) before speaking.\n- Streaming TTS strips markdown, emoji, and <think> blocks.\n- Desktop voice conversation uses per-reply speech WebSocket for overlapping speech and generation.\n- Barge-in during LLM generation interrupts the in-flight turn. The interjection becomes the next message.\n- Barge-in during playback cuts playback and submits what was said.\n- Typing or pressing the record key stops playback instantly.\n- Saying a stop phrase mid-generation interrupts the turn and ends the voice chat. Mid-playback, it cuts speech and ends the chat.\n- The agent knows it was interrupted and can react naturally.\n- Noise floor is calibrated against the quiet room at turn start, not against playback.\n- Recording auto-restarts after a reply unless stopped by Ctrl+B or three silent cycles.\n- If no speech is detected for 15 seconds, recording stops automatically.\n- The record key is configurable via voice.record_key in config.yaml.\n- Beep sounds can be disabled via voice.beep_enabled: false.\n- Silence detection: speech is confirmed after audio above RMS threshold (200) for at least 0.3 seconds. End is detected after 3.0 seconds of continuous silence.\n- Both silence_threshold and silence_duration are configurable in config.yaml.\n- Stop phrases list can be set to [] to disable voice end-by-phrase.\n- Voice chat also ends on its own after three consecutive silent cycles.\n- Typing a bare stop phrase while voice chat is active ends the chat instead of sending to the agent.\n- Outside a voice chat, typed "stop" is an ordinary message.\n- TTS streaming works with every TTS provider.\n- Providers with chunked PCM API (ElevenLabs, OpenAI) stream raw audio for lowest latency.\n- Other providers synthesize and play each sentence as it completes.\n- The same TTS pipeline runs in classic CLI, TUI, and desktop app.\n- In desktop voice conversation, reply text is fed live into per-reply speech WebSocket as the model generates.\n- One socket and one audio clock per reply, no per-sentence connection gaps.\n- Hallucination filter uses 26 known phrases and a regex pattern for repetitive variations.\n- Gateway voice reply works for both Telegram and Discord messaging platforms.\n- Discord supports two modes: DM (no mention needed) and Server Channel (@mention required by default).\n- DM is recommended for personal use.\n- In server channels, select the bot user from the mention popup, not the role with the same name.\n- Voice mode commands work in both Telegram and Discord (DMs and text channels).\n- Voice mode has three states: off (text only), voice_only (reply only to voice messages), all (reply to all messages).\n- Discord voice channel transcripts appear in the text channel where /voice join was issued.\n- Agent responses are sent as text in the channel and spoken in the VC.\n- Only users in DISCORD_ALLOWED_USERS can interact via voice in VC. Others are silently ignored.\n- DISCORD_ALLOWED_USERS can be numeric IDs or usernames. Usernames require the Server Members Intent.\n- The bot needs a SPEAKING event from Discord to map audio. Start speaking within a few seconds of joining.\n- If the bot doesn't hear you, check DISCORD_ALLOWED_USERS, mute status, and SPEAKING event.\n- If the bot hears but doesn't respond, verify STT availability and LLM configuration.\n- If the bot responds in text but not voice, check TTS provider, API key, quota, and logs.\n- Gateway logs are at ~/.hermes/logs/gateway.log.\n- TUI forwards crash-forensic logs to ~/.hermes/logs/.\n- STT can be disabled (enabled: false) to pass the audio file path to the agent for custom pipelines.\n- Local STT model options: tiny, base, small, medium, large-v3.\n- Optional ISO-639-1 language hint for STT. Blank = auto-detect or use HERMES_LOCAL_STT_LANGUAGE if set.\n- Groq STT model can be overridden via STT_GROQ_MODEL env var.\n- OpenAI STT model can be overridden via STT_OPENAI_MODEL env var.\n- Custom Groq endpoint via GROQ_BASE_URL env var.\n- Custom OpenAI STT endpoint via STT_OPENAI_BASE_URL env var.\n- ElevenLabs voice and model are configurable in config.yaml.\n- OpenAI TTS model and voice are configurable. base_url can be overridden for self-hosted or compatible endpoints.\n- OpenAI TTS text_to_speech tool accepts an optional instructions argument for tone, emotion, pacing, accent, and whispering.\n- The instructions argument also routes to OpenAI-compatible voice-design servers via tts.openai.base_url.\n- NeuTTS config includes ref_audio, ref_text, model, and device.\n- Discord permissions integer for text and voice: 309240908864.\n- Discord privileged gateway intents: Presence, Server Members (conditional), Message Content.\n- Opus codec must be installed on the machine running the gateway.\n- The bot auto-loads Opus from /opt/homebrew/lib/libopus.dylib (macOS) or libopus.so.0 (Linux).\n- Discord voice channel requires the user to be in VC before /voice join.\n- The bot joins the same VC the user is in.\n- Voice channel silence detection: 1.5 seconds of silence after at least 0.5 seconds of speech triggers processing.\n- The bot listens to each user's audio stream independently.\n- Text channel integration: transcripts and agent responses appear in the text channel where /voice join was issued.\n- Echo prevention: the bot pauses the audio listener while playing TTS replies.\n- Access control: only users in DISCORD_ALLOWED_USERS can interact via voice.\n- Gateway setup wizard: hermes gateway setup.\n- Nous Portal subscription provides LLM and OpenAI TTS via the Tool Gateway. No separate OpenAI key is needed.\n- On a fresh install, hermes setup --portal wires up Nous Portal at once.\n- ~/.hermes/ directory and default config.yaml are created automatically on the first hermes run.\n- Only ~/.hermes/.env needs manual creation for API keys.\n- If faster-whisper is installed, voice mode works with zero API keys for STT.\n- Model downloads automatically on first use (~150 MB for base).\n- Edge TTS and NeuTTS work without any API key.\n- ElevenLabs requires an API key.\n- OpenAI TTS requires VOICE_TOOLS_OPENAI_KEY.\n- Groq Whisper requires GROQ_API_KEY.\n- STT provider priority (automatic fallback): local > groq > openai.\n- TTS provider fallback: Edge TTS is the default free fallback.\n- For OpenAI TTS, the text_to_speech tool accepts an optional instructions argument for voice design.\n- The instructions argument also routes to OpenAI-compatible voice-design servers via tts.openai.base_url.\n- NeuTTS uses the tts.neutts config block.\n- Troubleshooting: "No audio device found" means PortAudio is not installed.\n- Docker users need host audio socket access for CLI voice mode.\n- The bot doesn't respond in Discord server channels: check @mention, use DM, or set DISCORD_REQUIRE_MENTION=false.\n- The bot joins VC but doesn't hear: check DISCORD_ALLOWED_USERS, mute status, SPEAKING event.\n- The bot hears but doesn't respond: check STT, LLM, and gateway logs.\n- The bot responds in text but not voice: check TTS provider, API key, quota, logs.\n- Whisper returns garbage: use a quieter environment, adjust silence_threshold, try a different STT model." }