Hermes Agent Fallback Providers: Staying Up When a Provider Fails

hermes fallback

fallback_providers:
  - provider: openrouter
    model: anthropic/claude-sonnet-4

fallback_providers:
  - provider: custom
    model: my-local-model
    base_url: http://localhost:8000/v1
    key_env: MY_LOCAL_KEY # env var name containing the API key

auxiliary:
  vision:
    provider: "auto" # auto | openrouter | nous | codex | main | anthropic
    model: "" # e.g. "openai/gpt-4o"
    base_url: "" # direct endpoint (takes precedence over provider)
    api_key: "" # API key for base_url

  web_extract:
    provider: "auto"
    model: ""

  compression:
    provider: "auto"
    model: ""
    fallback_chain: # optional, task-specific fallback policy
      - provider: openrouter
        model: inclusionai/ring-2.6-1t:free

  skills_hub:
    provider: "auto"
    model: ""

  mcp:
    provider: "auto"
    model: ""

auxiliary:
  compression:
    provider: main # Same provider options as other auxiliary tasks
    model: google/gemini-3-flash-preview
    base_url: null # Custom OpenAI-compatible endpoint

fallback_providers:
  - provider: openrouter
    model: anthropic/claude-sonnet-4
    # base_url: http://localhost:8000/v1 # Optional custom endpoint

delegation:
  provider: "openrouter" # override provider for all subagents
  model: "google/gemini-3-flash-preview" # override model
  # base_url: "http://localhost:1234/v1" # or use a direct endpoint
  # api_key: "local-key"

cronjob (
  action = "create",
  schedule = "every 2h",
  prompt = "Check server status",
  provider = "openrouter",
  model = "google/gemini-3-flash-preview"
)

How It Works

Primary Model Fallback Trigger Conditions

The fallback activates automatically when the primary model fails with:

• Rate limits (HTTP 429) after exhausting retry attempts
• Server errors (HTTP 500, 502, 503) after exhausting retry attempts
• Auth failures (HTTP 401, 403) immediately (no point retrying)
• Not found (HTTP 404) immediately
• Invalid responses when the API returns malformed or empty responses repeatedly

When triggered, Hermes:

1. Resolves credentials for the fallback provider
2. Builds a new API client
3. Swaps the model, provider, and client in-place
4. Resets the retry counter and continues the conversation

Auxiliary Task Auto-Detection Chain

When a task's provider is set to "auto" (the default), Hermes first tries the main provider plus main model for that auxiliary task. If that route is unavailable or later fails with a capacity-style error, Hermes honors user-configured fallback policy before using the built-in discovery chain. The resolution order is:

Main provider + main model -> auxiliary.<task>.fallback_chain ->
fallback_providers / fallback_model -> built-in auxiliary discovery chain

The task-specific chain is most precise and wins when present. The top-level fallback_providers chain is the same policy the main agent uses, so free-only or same-provider fallback rules apply to auxiliary tasks on auto as well.

Built-in text discovery chain (compression, web extract, title generation, and others):

OpenRouter -> Nous Portal -> Custom endpoint -> Codex OAuth ->
API-key providers (z.ai, Kimi, MiniMax, Xiaomi MiMo, Hugging Face, Anthropic) -> give up

Built-in vision discovery chain:

Main provider (if vision-capable) -> OpenRouter -> Nous Portal ->
Codex OAuth -> Anthropic -> Custom endpoint -> give up

Those built-in chains are a convenience fallback for users who have not declared a task-specific or main fallback policy.

Auxiliary Capacity-Error Fallback

When you set an explicit auxiliary provider (for example, auxiliary.vision.provider: glm), Hermes treats that as your preferred choice. But if the provider literally cannot serve the request because of a capacity error (HTTP 402 payment required, HTTP 429 daily-quota exhaustion, connection failure), Hermes falls back through a layered chain instead of failing silently:

1. Primary aux provider (the one you configured, tried first, always)
2. auxiliary.<task>.fallback_chain (your per-task override list, if you wrote one)
3. Main agent provider + model (last-resort safety net, always tried, even if you did not write a chain)
4. Warn + re-raise (if every layer fails, Hermes logs Auxiliary <task>: all fallbacks exhausted at WARNING level and re-raises the original error)

Transient HTTP 429 rate limits with Retry-After headers are treated as request constraints, not capacity problems. They respect your explicit provider choice and do not trigger the fallback ladder. Only daily or monthly quota exhaustion, payment errors, and connection failures bypass the explicit-provider gate.

For users on provider: auto (no explicit aux provider), the existing auto-detection chain runs in place of steps 2-3. Its first step is already the main agent model, so auto users get the same outcome with zero config.

Provider Quota Errors That Trigger Fallback

Hermes recognizes these error strings as capacity-equivalent to 402 credit exhaustion (not transient rate limits):

• Bedrock / LiteLLM: Too many tokens per day, daily limit, tokens per day
• Vertex AI / GCP: quota exceeded, resource exhausted, RESOURCE_EXHAUSTED
• Generic: daily quota, quota_exceeded

If your provider returns a different phrase for daily-quota exhaustion and Hermes does not trigger fallback, that is a bug. Open an issue with the exact error string.

Direct Endpoint Override

For any auxiliary task, setting base_url bypasses provider resolution entirely and sends requests directly to that endpoint:

auxiliary:
  vision:
    base_url: "http://localhost:1234/v1"
    api_key: "local-key"
    model: "qwen2.5-vl"

base_url takes precedence over provider. Hermes uses the configured api_key for authentication, falling back to OPENAI_API_KEY if not set. It does not reuse OPENROUTER_API_KEY for custom endpoints.

Context Compression Fallback

Context compression uses the auxiliary.compression config block to control which model and provider handles summarization:

auxiliary:
  compression:
    provider: "auto" # auto | openrouter | nous | main
    model: "google/gemini-3-flash-preview"

Older configs with compression.summary_model / compression.summary_provider / compression.summary_base_url are automatically migrated to auxiliary.compression.* on first load (config version 17).

If no provider is available for compression, Hermes drops middle conversation turns without generating a summary rather than failing the session.

Requirements and Limitations

Prerequisites

• Hermes Agent must be installed and configured with at least one primary provider.
• For each fallback provider you want to use, the corresponding environment variable or authentication method must be set up. See the Supported Providers table for exact requirements.
• The fallback_providers list must be configured in ~/.hermes/config.yaml or via the hermes fallback interactive manager.

Where Fallback Works

Context	Fallback Supported
CLI sessions	Yes
Messaging gateway (Telegram, Discord, etc.)	Yes
Subagent delegation	Yes (subagents inherit the parent fallback chain)
Cron jobs	Yes (cron agents inherit configured fallback providers)
Auxiliary tasks on provider: auto	Yes (try per-task fallback, then the main fallback chain before built-in aux discovery)

Limitations

• Fallback is turn-scoped. Each new user message starts with the primary model restored. If the primary fails mid-turn, fallback activates for that turn only. On the next message, Hermes tries the primary again.
• Within a single turn, fallback activates at most once. If the fallback also fails, normal error handling takes over (retries, then error message). This prevents cascading failover loops within a turn.
• Fallback resets the prompt cache. When fallback fires, the new provider:model has no cached prefix for the conversation, so the next request re-reads the entire history at full input-token price. A long session that bounces between providers can cost noticeably more than one that stays put.
• There are no environment variables for the primary fallback chain. Configure it exclusively through config.yaml or hermes fallback.
• "main" is not a valid value for your top-level model.provider. It is only valid for auxiliary task configuration.
• For delegation and cron jobs, there is no automatic fallback. Only provider override is supported.
• If your provider returns a different phrase for daily-quota exhaustion and Hermes does not trigger fallback, that is a bug. Open an issue with the exact error string.

Troubleshooting

Fallback Does Not Trigger

If fallback does not activate when you expect it to, check the following:

• Ensure the fallback_providers list is correctly formatted in config.yaml. Each entry requires both provider and model. Entries missing either field are ignored.
• Verify that the fallback provider's credentials are set correctly. For example, if using OpenRouter as a fallback, ensure OPENROUTER_API_KEY is set.
• For auxiliary tasks with an explicit provider, transient HTTP 429 rate limits with Retry-After headers are treated as request constraints, not capacity problems. They do not trigger the fallback ladder. Only daily or monthly quota exhaustion, payment errors, and connection failures bypass the explicit-provider gate.
• If you are using a custom endpoint, ensure base_url and key_env (or api_key) are correctly configured.

Fallback Triggers But Fails

If the fallback provider is selected but the request still fails:

• Check that the fallback provider's credentials are valid and have not expired.
• Verify that the model specified in the fallback entry exists and is accessible with the provided credentials.
• If using a custom endpoint, ensure the endpoint is running and reachable.
• Within a single turn, fallback activates at most once. If the fallback also fails, normal error handling takes over (retries, then error message). The next user message will try the primary model again.

Legacy Configuration Not Working

If you have an older config with fallback_model (singular) and it is not being used:

• fallback_providers (plural, list) takes priority when both are set.
• Run hermes fallback to migrate legacy config to the current format. The command writes the current fallback_providers key and migrates legacy config on write.

Context Compression Not Working

If context compression is failing and no summary is generated:

• Check the auxiliary.compression configuration block. Ensure provider and model are set correctly.
• If no provider is available for compression, Hermes drops middle conversation turns without generating a summary rather than failing the session. This is expected behavior.
• Older configs with compression.summary_model / compression.summary_provider / compression.summary_base_url are automatically migrated to auxiliary.compression.* on first load (config version 17). If migration does not happen, manually update your config.

Quota Error Not Triggering Fallback

If your provider returns a daily-quota exhaustion error but Hermes does not trigger fallback:

• Check the exact error string. Hermes recognizes these as capacity-equivalent to 402 credit exhaustion: Too many tokens per day, daily limit, tokens per day, quota exceeded, resource exhausted, RESOURCE_EXHAUSTED, daily quota, quota_exceeded.
• If your provider returns a different phrase, that is a bug. Open an issue with the exact error string.

Auxiliary Task All Fallbacks Exhausted

If you see a log message like Auxiliary <task>: all fallbacks exhausted at WARNING level:

• This means every layer of the fallback chain (primary aux provider, per-task fallback chain, main agent provider + model) has failed.
• The original error is re-raised. Check the error details to determine which provider failed and why.
• Verify that at least one provider in the chain has valid credentials and is accessible.

Examples

OpenRouter as Fallback for Anthropic Native

model:
  provider: anthropic
  default: claude-sonnet-4-6

fallback_providers:
  - provider: openrouter
    model: anthropic/claude-sonnet-4

Nous Portal as Fallback for OpenRouter

model:
  provider: openrouter
  default: anthropic/claude-opus-4

fallback_providers:
  - provider: nous
    model: nous-hermes-3

Local Model as Fallback for Cloud

fallback_providers:
  - provider: custom
    model: llama-3.1-70b
    base_url: http://localhost:8000/v1
    key_env: LOCAL_API_KEY

Codex OAuth as Fallback

fallback_providers:
  - provider: openai-codex
    model: gpt-5.3-codex

Per-Task Fallback Chain for Vision

auxiliary:
  vision:
    provider: glm
    model: glm-4v-flash
    fallback_chain:
      - provider: openrouter
        model: google/gemini-3-flash-preview
      - provider: nous
        model: anthropic/claude-sonnet-4

  compression:
    provider: openrouter
    fallback_chain:
      - provider: openai
        model: gpt-4o-mini
        timeout: 240 # optional - this candidate's own deadline (seconds)