Configuration Parameters

Model Parameters

• model.context_length - Total context window in tokens (input plus output). Overrides auto-detection.
• model.max_tokens - Maximum output tokens per response. Unrelated to context_length.
• model.supports_vision - Boolean flag to enable native vision support for custom endpoint models.
• model.base_url - Base URL for custom endpoint (set in config.yaml, not .env). Required for custom endpoints.
• model.api_key - API key for custom endpoint (can be left empty for local servers).
• model.lmstudio_load_mode - LM Studio loading mode: explicit (default) or jit (Just-In-Time).

Named Custom Provider Parameters

• providers.<name>.api - Base URL for a named custom provider. Required.
• providers.<name>.key_env - Environment variable name containing the API key.
• providers.<name>.transport - API transport mode: chat_completions, anthropic_messages, or codex_responses.
• providers.<name>.default_model - Default model ID.
• providers.<name>.extra_body - Additional JSON fields to merge into chat-completions requests.
• providers.<name>.extra_headers - Additional HTTP headers to send with requests.
• providers.<name>.ssl_ca_cert - Path to CA certificate for SSL verification.
• providers.<name>.ssl_verify - Boolean to enable or disable SSL verification.
• providers.<name>.enabled - Boolean to hide a provider entry without deleting it (default: true).

Fallback and Routing Parameters

• fallback_providers - List of fallback provider entries, each with provider, model, and optional base_url and api_mode.
• fallback_model - Legacy single fallback provider dictionary with provider and model.
• provider_routing.sort - OpenRouter routing sort order: price (default), throughput, or latency.
• provider_routing.only - List of providers to exclusively use for OpenRouter routing.
• provider_routing.ignore - List of providers to skip for OpenRouter routing.
• provider_routing.order - Ordered list of providers to try for OpenRouter routing.
• provider_routing.require_parameters - Boolean to only use providers supporting all request parameters.
• provider_routing.data_collection - String deny to exclude providers that may store or train on data.
• openrouter.min_coding_score - Minimum coding quality score (0.0 to 1.0) for the Pareto Code router. Default is 0.65.

AWS Bedrock Parameters

• bedrock.region - AWS region for Bedrock (config.yaml).
• bedrock.profile - AWS profile for Bedrock (config.yaml).
• bedrock.discovery - Boolean to auto-discover region from IAM.
• bedrock.guardrail - Optional Bedrock Guardrails configuration with guardrail_identifier and guardrail_version.

Google Vertex AI Parameters

• vertex.project_id - GCP project ID for Vertex AI (config.yaml).
• vertex.region - GCP region for Vertex AI (config.yaml).

Provider-Specific Details

OpenRouter

Set OPENROUTER_API_KEY in ~/.hermes/.env. Run hermes chat --provider openrouter --model anthropic/claude-sonnet-4. Provider routing can sort by price, throughput, or latency. The Pareto Code router uses openrouter/pareto-code as the model name with a configurable min_coding_score.

Anthropic

Two authentication paths exist. The API key path uses ANTHROPIC_API_KEY and is pay-per-token, independent of any Claude subscription. The OAuth path requires a Claude Max plan with purchased extra usage credits. Claude Pro subscribers cannot use the OAuth path. Run hermes chat --provider anthropic --model claude-sonnet-4-6.

GitHub Copilot

Authentication uses COPILOT_GITHUB_TOKEN, GH_TOKEN, or GITHUB_TOKEN in priority order, or OAuth through hermes model. The GitHub Copilot API does not support classic Personal Access Tokens (ghp_*). Supported token types are gho_*, github_pat_*, and ghu_*. ACP mode requires the GitHub Copilot CLI in PATH and an existing copilot login session. Run hermes chat --provider copilot --model gpt-5.4.

Fireworks AI

Requires FIREWORKS_API_KEY. Uses native slash-form catalog IDs, for example accounts/fireworks/models/kimi-k2p6. The base URL must be configured via model.base_url in config.yaml, not in .env.

NovitaAI

Requires NOVITA_API_KEY. The base URL can be overridden with NOVITA_BASE_URL. Example model: moonshotai/kimi-k2.5.

z.ai / GLM

Requires GLM_API_KEY. Endpoint auto-detection probes multiple endpoints, so no manual GLM_BASE_URL is needed. Example model: glm-5.

Kimi/Moonshot

Requires KIMI_API_KEY for the international endpoint or KIMI_CN_API_KEY for the China endpoint. The base URL can be overridden with KIMI_BASE_URL. Example model: kimi-for-coding.

MiniMax

Requires MINIMAX_API_KEY for the global endpoint or MINIMAX_CN_API_KEY for the China endpoint. The base URL can be overridden with MINIMAX_BASE_URL or MINIMAX_CN_BASE_URL. Example model: MiniMax-M2.7.

xAI (Grok)

Requires XAI_API_KEY for the API key path. The OAuth path requires a SuperGrok or X Premium+ subscription. Prompt caching activates automatically when an xAI endpoint is detected and a session ID is available. A web search backend is also available. Example model: grok-4-fast-reasoning.

Qwen Cloud (Alibaba)

Requires DASHSCOPE_API_KEY. The base URL can be overridden with DASHSCOPE_BASE_URL. Example model: qwen3.5-plus.

AWS Bedrock

Uses AWS credentials from the standard boto3 chain. The AWS_PROFILE and AWS_REGION environment variables can be used. A non-default regional endpoint can be set with BEDROCK_BASE_URL. Example model: us.anthropic.claude-sonnet-4-6.

Google Vertex AI

Configure through hermes model by selecting "Google Vertex AI", then providing the project, region, and model. Uses OAuth2. Alternatively, set VERTEX_CREDENTIALS_PATH, VERTEX_PROJECT_ID, and VERTEX_REGION.

Ollama Cloud

Requires OLLAMA_API_KEY. The model catalog is fetched dynamically from ollama.com/v1/models and cached for one hour. Uses model:tag notation, for example qwen3-coder:480b-cloud. Do not use dashes in the model name.

Named Custom Provider Example (Together AI)

Add the following to ~/.hermes/config.yaml:

providers:
  together:
    api: https://api.together.xyz/v1
    key_env: TOGETHER_API_KEY

Then set model.provider to custom:together and model.default to the desired model ID.

Fallback Provider Configuration

Fallback is configured exclusively through config.yaml or interactively via hermes fallback. Activation is one-shot per session, meaning the fallback chain is tried entry-by-entry only on the first failure. The conversation is preserved across the fallback.

Example fallback configuration:

fallback_providers:
  - provider: openrouter
    model: anthropic/claude-sonnet-4
  - provider: anthropic
    model: claude-sonnet-4

Context Length Configuration

Context length auto-detection is provider-aware, meaning the same model can have different limits on different providers. The detection pulls from multiple sources in order: configuration, endpoint API, model metadata, and a community registry.

Set context_length manually when using Ollama with custom num_ctx, when limiting context below the maximum, or when behind a proxy without a /v1/models endpoint. The model.max_tokens parameter controls response length only and is unrelated to conversation history length, which is governed by context_length.

Tool Calling on Local Servers

Tool calling on local servers requires server-side flags. For vLLM, use --enable-auto-tool-choice and --tool-call-parser. For SGLang, use --tool-call-parser. For llama.cpp, use the --jinja flag. For Ollama, tool calling is supported through the OpenAI-compatible API. For LM Studio, tool calling support depends on the loaded model.

If tool calls appear as text instead of executing, the server likely lacks the required tool calling flags.

Vision-Capable Models

Vision-capable models are supported natively through the supports_vision flag. For custom endpoints, set model.supports_vision: true in config.yaml.

Auxiliary Model Routing

Some tools, such as vision, web summarization, and MoA, use a separate auxiliary model. The default setting auxiliary.*.provider: 'auto' routes to the main chat model. This can be overridden to use a different model for specific tasks.

Nous Tool Gateway

The Nous Tool Gateway provides web search, image generation, TTS, and browser automation for paid subscribers. It requires a paid Nous Portal subscription.

OpenAI Codex Provider

The OpenAI Codex provider stores credentials in ~/.hermes/auth.json and can import from ~/.codex/auth.json. Token refresh failures (HTTP 4xx, invalid_grant) mark the refresh token as dead. Run hermes auth add openai-codex for a fresh login.

Nous Portal Authentication

Nous Portal requests carry a client=hermes-client-v<version> tag that is auto-aligned to the installed release. JWT authentication is automatic, and revoked refresh tokens are quarantined to avoid replay loops.

ClawRouter

ClawRouter requires a USDC-funded wallet on Base or Solana for payment. If payment issues occur, run npx @blockrun/clawrouter doctor to check wallet status.

Legacy Configuration Migration

The legacy LLM_MODEL environment variable is removed. The config.yaml file is the single source of truth for model and endpoint configuration. Legacy custom_providers: list format is auto-migrated to the providers: dictionary format (config v12). Run hermes update to trigger auto-migration if needed.

The model key alias in config can be either default: or model:; both work identically.

Failure Modes and Troubleshooting

Connection Issues

• Connection refused from WSL2 to a Windows-hosted model server in NAT mode. Use mirrored mode or the Windows host IP.
• WSL2 mirrored mode not working. Add a Hyper-V firewall rule: Set-NetFirewallHyperVVMSetting.
• Windows Firewall blocking connections. Add an inbound rule for the model server port.

Tool Calling Issues

• Tool calls appear as text instead of executing. The server lacks tool calling flags. Add --jinja for llama.cpp, --enable-auto-tool-choice and --tool-call-parser for vLLM, or --tool-call-parser for SGLang.

Context and Response Issues

• Model forgets context or gives incoherent responses. The context window is too small. Ensure it is at least 64,000 tokens.
• Context limit shows 2,048 tokens at startup. The server reports a low value or none. Set context_length explicitly in config.yaml.
• Responses cut off mid-sentence. The max_tokens value is too low (SGLang defaults to 128) or context is exhausted.
• Ollama context too small. Set the OLLAMA_CONTEXT_LENGTH environment variable, use a systemd override, or configure a Modelfile.

Authentication Issues

• GitHub Copilot authentication fails with a ghp_* token. Use OAuth by running hermes model to get a gho_* token.
• Anthropic OAuth fails without Claude Max plus extra usage credits. Use ANTHROPIC_API_KEY instead.
• OpenAI Codex token refresh fails with a terminal error. Run hermes auth add openai-codex for a fresh login.

Model and Provider Issues

• xAI model retired after 2026-05-15. Run hermes migrate xai --apply.
• SGLang responses truncated. Set --default-max-tokens on the server or max_tokens in the request.
• llama.cpp tool calls not working. Add the --jinja flag.
• LM Studio model not loading. Check the context length setting and model compatibility.
• Self-hosted Firecrawl failing on protected sites. This is expected, as self-hosted mode has no anti-bot bypass.
• ClawRouter payment issues. Run npx @blockrun/clawrouter doctor to check wallet status.
• Unknown provider warnings in hermes doctor. Ensure the provider name is correct and the configuration is valid.
• Provider /v1/models endpoint unreachable (for example, Perplexity). The model is persisted with a warning; set the model ID manually.
• Legacy config format not migrated. Run hermes update to auto-migrate to config v12.

Qwen Reasoning Models

For Qwen reasoning models served by vLLM, ensure the final user-visible response appears in the content field. Disable the reasoning parser or set enable_thinking: false via extra_body.

Examples

OpenRouter

Set OPENROUTER_API_KEY in ~/.hermes/.env, then run:

hermes chat --provider openrouter --model anthropic/claude-sonnet-4

Anthropic API Key

Export ANTHROPIC_API_KEY, then run:

hermes chat --provider anthropic --model claude-sonnet-4-6

GitHub Copilot

Run:

hermes chat --provider copilot --model gpt-5.4

This uses COPILOT_GITHUB_TOKEN or OAuth.

Fireworks AI

Requires FIREWORKS_API_KEY. Run:

hermes chat --provider fireworks --model accounts/fireworks/models/kimi-k2p6

NovitaAI

Requires NOVITA_API_KEY. Run:

hermes chat --provider novita --model moonshotai/kimi-k2.5

z.ai/GLM

Requires GLM_API_KEY. Run:

hermes chat --provider zai --model glm-5

Kimi/Moonshot

Requires KIMI_API_KEY. Run:

hermes chat --provider kimi-coding --model kimi-for-coding

MiniMax

Requires MINIMAX_API_KEY. Run:

hermes chat --provider minimax --model MiniMax-M2.7

xAI (Grok)

Requires XAI_API_KEY. Run:

hermes chat --provider xai --model grok-4-fast-reasoning

Qwen Cloud

Requires DASHSCOPE_API_KEY. Run:

hermes chat --provider alibaba --model qwen3.5-plus

AWS Bedrock

Uses AWS credentials. Run:

hermes chat --provider bedrock --model us.anthropic.claude-sonnet-4-6

Google Vertex AI

Run hermes model, select "Google Vertex AI", then provide the project, region, and model. Uses OAuth2.

Ollama Local

Run hermes model, select "Custom endpoint", enter http://localhost:11434/v1, leave the API key blank, and enter the model name.

vLLM

Run hermes model, select "Custom endpoint", enter http://localhost:8000/v1, optionally provide an API key, and enter the model name.

SGLang

Run hermes model, select "Custom endpoint", enter http://localhost:30000/v1, and enter the model name.

llama.cpp

Run hermes model, select "Custom endpoint", enter http://localhost:8080/v1, leave the API key blank, and enter the model name.

LM Studio

Run hermes model, select "LM Studio", accept the default URL http://localhost:1234/v1, and pick the discovered model.

Named Custom Provider (Together AI)

Add to config.yaml:

providers:
  together:
    api: https://api.together.xyz/v1
    key_env: TOGETHER_API_KEY

Set model.provider to custom:together and model.default to the desired model ID.

Fallback Providers

Add to config.yaml:

fallback_providers:
  - provider: openrouter
    model: anthropic/claude-sonnet-4
  - provider: anthropic
    model: claude-sonnet-4

OpenRouter Pareto Code

Set in config.yaml:

model:
  model: openrouter/pareto-code
openrouter:
  min_coding_score: 0.65

Self-Hosted Firecrawl

Clone the repository, set USE_DB_AUTHENTICATION=false, run docker compose up -d, then run:

hermes config set FIRECRAWL_API_URL http://localhost:3002

xAI Model Migration

Preview replacements:

hermes migrate xai

Apply with backup:

hermes migrate xai --apply

Additional Notes

The hermes model command (run from the terminal) is for full provider setup. The /model command (run inside a chat session) only switches between already-configured providers. To add a new provider, exit the chat session with Ctrl+C or /quit, run hermes model, then start a new session.

Fireworks AI uses native slash-form catalog IDs, for example accounts/fireworks/models/kimi-k2p6. The Fireworks AI base URL must be configured via model.base_url in config.yaml, not in .env.

The Z.AI / GLM endpoint auto-detection probes multiple endpoints, so no manual GLM_BASE_URL is needed.

Ollama Cloud model catalog is fetched dynamically from ollama.com/v1/models and cached for one hour. Ollama Cloud uses model:tag notation, for example qwen3-coder:480b-cloud. Do not use dashes.

Ollama local server does not use context length from the OpenAI-compatible API. It must be set server-side or via a Modelfile. Ollama defaults to very low context lengths: 4,096 tokens for systems with less than 24 GB VRAM, 32,768 tokens for 24-48 GB, and 256,000 tokens for systems with more than 48 GB VRAM. Hermes requires at least 64,000 tokens.

vLLM tool calling requires explicit flags: --enable-auto-tool-choice and --tool-call-parser. vLLM supports human-readable sizes: 64k equals 64,000, and 64K equals 65,536.

SGLang defaults to 128 max output tokens. Set --default-max-tokens on the server or max_tokens in the request. SGLang tool calling requires the --tool-call-parser flag.

llama.cpp requires the --jinja flag for tool calling support. The context length flag -c defaults to 0, which reads the model's training context. This may cause out-of-memory errors for 128k+ models, so set it explicitly to at least 64,000. Parallel slots (-np) divide the context among slots, and each slot must have at least 64,000 tokens.

LM Studio context length must be set in the GUI (gear icon) or CLI (lms load --context-length). Just-In-Time loading mode (jit) skips explicit preload and is set via hermes config set model.lmstudio_load_mode jit.

WSL2 default NAT mode means localhost inside WSL2 refers to the Linux VM, not the Windows host. Use mirrored mode or the Windows host IP. Mirrored mode requires Windows 11 22H2+ and may need a Hyper-V firewall rule (Set-NetFirewallHyperVVMSetting). In NAT mode, the model server on Windows must bind to 0.0.0.0, not 127.0.0.1.

Self-hosted Firecrawl uses basic fetch with Playwright and has no advanced anti-bot bypass. Search uses DuckDuckGo instead of Google. It requires approximately 4-8 GB RAM across 5 Docker containers.

ClawRouter requires a USDC-funded wallet on Base or Solana for payment.

The legacy LLM_MODEL environment variable is removed. The config.yaml file is the single source of truth for model and endpoint configuration. Legacy custom_providers: list format is auto-migrated to the providers: dictionary format (config v12).

Fallback is configured exclusively through config.yaml or interactively via hermes fallback. Fallback activation is one-shot per session, and the chain is tried entry-by-entry.

Context length auto-detection is provider-aware, meaning the same model can have different limits on different providers. Set context_length manually when using Ollama with custom num_ctx, when limiting context below the maximum, or when behind a proxy without a /v1/models endpoint.

The model.max_tokens parameter controls response length only and is unrelated to conversation history length, which is governed by context_length.

For Qwen reasoning models served by vLLM, ensure the final user-visible response appears in the content field. Disable the reasoning parser or set enable_thinking: false via extra_body.

The model key alias in config can be either default: or model:; both work identically.

The OpenRouter Pareto Code router uses a minimum coding score of 0.65 by default, but you can set it to any value between 0.0 and 1.0. A score of 100% indicates perfect coding quality, while 0% means no coding capability is required. Setting it to 0% disables the filter entirely, allowing any provider to be used regardless of coding performance.

When configuring context length, note that the maximum supported value is 200000 tokens for most providers, though some may support more. Setting context length to 0% of the maximum (effectively 0 tokens) is not allowed and will result in an error. The minimum recommended context length is 64,000 tokens, and setting it to 100% of the maximum (200000 tokens) is supported but may require significant VRAM or RAM.

The cost per 1,000 tokens for some providers can be as low as $0.10, making them suitable for high-volume or budget-constrained workloads. Always check the provider's pricing page for the most current rates, as prices can change without notice.