Hermes Agent AI Providers: Configuring Inference Endpoints

This integration connects Hermes Agent, an open-source AI agent runtime by Nous Research, to over 30 inference providers ranging from cloud APIs (OpenRouter, Anthropic, OpenAI) to self-hosted endpoints (Ollama, vLLM, LM Studio) and managed platforms (AWS Bedrock, Google Vertex AI, Azure AI Foundry). Once wired up, an agent can use any supported large language model for chat, tool use, code generation, and auxiliary tasks like vision or summarization, with automatic failover, prompt caching, and credential rotation.

What It Does

• Provides a unified interface to 30+ LLM providers through a single CLI command (hermes model) or configuration file.
• Supports multiple authentication methods: API keys, OAuth device code flow, browser-based PKCE login, service account JSON, and AWS credential chain.
• Enables dynamic provider switching during a chat session via the /model command.
• Routes auxiliary tasks (vision, web summarization, mixture-of-agents) to the main model or a separate cheaper/faster model.
• Includes built-in support for prompt caching on xAI endpoints, automatic endpoint detection for Z.AI/GLM, and credential recovery on HTTP 401.
• Offers a unified subscription gateway (Nous Portal) that covers 300+ models and tool services (web search, image generation, TTS, browser automation) with a single OAuth login.
• Supports fallback and failover configurations for production reliability.

Setup

Prerequisites

• Hermes Agent installed (see the Hermes Agent installation guide).
• At least one inference provider configured. Hermes will not start without a provider.
• For cloud providers: an API key or OAuth credentials from the provider's platform.
• For self-hosted providers: a running local or remote endpoint (e.g., Ollama, vLLM, LM Studio).

Two Commands for Model Management

Hermes provides two commands for managing providers and models:

Command	Where to run	What it does
hermes model	Your terminal (outside any session)	Full setup wizard, add providers, run OAuth, enter API keys, configure endpoints
/model	Inside a Hermes chat session	Quick switch between already-configured providers and models

If you are trying to switch to a provider you haven't set up yet (e.g., you only have OpenRouter configured and want to use Anthropic), you need hermes model, not /model. Exit your session first (Ctrl+C or /quit), run hermes model, complete the provider setup, then start a new session.

Environment File

Most API-key-based providers store their credentials in ~/.hermes/.env. This file is a standard shell environment file. Create or edit it with your preferred text editor:

# ~/.hermes/.env
OPENROUTER_API_KEY=sk-or-v1-...
ANTHROPIC_API_KEY=sk-ant-...
FIREWORKS_API_KEY=...

Provider-Specific Setup

Nous Portal (Recommended)

Nous Portal is Nous Research's unified subscription gateway and the recommended way to run Hermes Agent. One OAuth login covers 300+ frontier agentic models (Claude, GPT, Gemini, DeepSeek, Qwen, Kimi, GLM, MiniMax, Grok) plus the Tool Gateway (web search, image generation, TTS, browser automation), billed against your Nous subscription instead of separate per-provider accounts.

# Fresh install, OAuth + provider + gateway in one command
hermes setup --portal

# Existing install, pick "Nous Portal" from the list
hermes model

# Inspect login + routing at any time
hermes portal info

Don't have a subscription yet? Get one at portal.nousresearch.com/manage-subscription.

Client identification: Every Portal request from Hermes Agent carries a client=hermes-client-v<version> tag (e.g., client=hermes-client-v0.13.0) auto-aligned to your installed release. This is sent on all Portal pathways, main chat loop, auxiliary calls, compression summarizer, web extraction, and lets Portal-side telemetry distinguish Hermes traffic from other clients. No config required; the tag updates automatically when you hermes update.

JWT auth (automatic): Hermes prefers scoped inference:invoke JWTs for Portal requests with the legacy opaque session-key path as a fallback. No configuration is required, credentials are managed by the OAuth flow and rotate transparently. Revoked refresh tokens are quarantined to avoid replay loops.

OpenAI Codex

The OpenAI Codex provider authenticates via device code (open a URL, enter a code). Hermes stores the resulting credentials in its own auth store under ~/.hermes/auth.json and can import existing Codex CLI credentials from ~/.codex/auth.json when present. No Codex CLI installation is required.

If a token refresh fails with a terminal error (HTTP 4xx, invalid_grant, revoked grant, etc.), Hermes marks the refresh token as dead and stops replaying it so you don't see a flood of identical auth failures. The next request surfaces a typed re-auth message instead. Run hermes auth add openai-codex (or hermes model → OpenAI Codex) to start a fresh device-code login; the quarantine clears on the next successful exchange.

GitHub Copilot

Hermes supports GitHub Copilot as a first-class provider with two modes:

copilot, Direct Copilot API (recommended). Uses your GitHub Copilot subscription to access GPT-5.x, Claude, Gemini, and other models through the Copilot API.

hermes chat --provider copilot --model gpt-5.4

Authentication options (checked in this order):

• COPILOT_GITHUB_TOKEN environment variable
• GH_TOKEN environment variable
• GITHUB_TOKEN environment variable
• gh auth token CLI fallback

If no token is found, hermes model offers an OAuth device code login, the same flow used by the Copilot CLI and opencode.

Token types: The Copilot API does not support classic Personal Access Tokens (ghp_*). Supported token types:

Type	Prefix	How to get
OAuth token	gho_	hermes model → GitHub Copilot → Login with GitHub
Fine-grained PAT	github_pat_	GitHub Settings → Developer settings → Fine-grained tokens (needs Copilot Requests permission)
GitHub App token	ghu_	Via GitHub App installation

If your gh auth token returns a ghp_* token, use hermes model to authenticate via OAuth instead.

Copilot auth behavior in Hermes: Hermes sends a supported GitHub token (gho_*, github_pat_*, or ghu_*) directly to api.githubcopilot.com and includes Copilot-specific headers (Editor-Version, Copilot-Integration-Id, Openai-Intent, x-initiator). On HTTP 401, Hermes now performs a one-shot credential recovery before fallback:

• Re-resolve token via the normal priority chain (COPILOT_GITHUB_TOKEN → GH_TOKEN → GITHUB_TOKEN → gh auth token)
• Rebuild the shared OpenAI client with refreshed headers
• Retry the request once

Some older community proxies use api.github.com/copilot_internal/v2/token exchange flows. That endpoint can be unavailable for some account types (returns 404). Hermes therefore keeps direct-token auth as the primary path and relies on runtime credential refresh + retry for robustness.

API routing: GPT-5+ models (except gpt-5-mini) automatically use the Responses API. All other models (GPT-4o, Claude, Gemini, etc.) use Chat Completions. Models are auto-detected from the live Copilot catalog.

copilot-acp, Copilot ACP agent backend. Spawns the local Copilot CLI as a subprocess:

hermes chat --provider copilot-acp --model copilot-acp
# Requires the GitHub Copilot CLI in PATH and an existing `copilot login` session

Permanent config:

model:
  provider: "copilot"
  default: "gpt-5.4"

Environment variables:

Variable	Description
COPILOT_GITHUB_TOKEN	GitHub token for Copilot API (first priority)
HERMES_COPILOT_ACP_COMMAND	Override the Copilot CLI binary path (default: copilot)
HERMES_COPILOT_ACP_ARGS	Override ACP args (default: --acp --stdio)

Anthropic (Native)

Use Claude models directly through the Anthropic API, no OpenRouter proxy needed. Supports three auth methods:

Requires Claude Max "extra usage" credits: When you authenticate via hermes model → Anthropic OAuth (or via hermes auth add anthropic --type oauth), Hermes routes as Claude Code against your Anthropic account. It only works if you're on a Claude Max plan and have purchased extra usage credits. The base Max plan allowance (the usage included in Claude Code by default) is not consumed by Hermes, only the extra/overage credits you've added on top are. Claude Pro subscribers cannot use this path.

If you don't have Max + extra credits, use an ANTHROPIC_API_KEY instead, requests are billed pay-per-token against that key's organization (standard API pricing, independent of any Claude subscription).

# With an API key (pay-per-token)
export ANTHROPIC_API_KEY=sk-ant-...
hermes chat --provider anthropic --model claude-sonnet-4-6

# Preferred: authenticate through `hermes model`
# Hermes will use Claude Code's credential store directly when available
hermes model

# Manual override with a setup-token (fallback / legacy)
export ANTHROPIC_TOKEN=...  # setup-token or manual OAuth token
hermes chat --provider anthropic

# Auto-detect Claude Code credentials (if you already use Claude Code)
hermes chat --provider anthropic  # reads Claude Code credential files automatically

When you choose Anthropic OAuth through hermes model, Hermes prefers Claude Code's own credential store over copying the token into ~/.hermes/.env. That keeps refreshable Claude credentials refreshable.

Or set it permanently:

model:
  provider: "anthropic"
  default: "claude-sonnet-4-6"

Aliases --provider claude and --provider claude-code also work as shorthand for --provider anthropic.

OpenRouter

# Requires: OPENROUTER_API_KEY in ~/.hermes/.env
hermes chat --provider openrouter --model openai/gpt-4o

Fireworks AI

# Requires: FIREWORKS_API_KEY in ~/.hermes/.env
hermes chat --provider fireworks --model accounts/fireworks/models/kimi-k2p6

Fireworks uses its native slash-form catalog IDs, such as accounts/fireworks/models/kimi-k2p6. Run hermes model, choose Fireworks AI, and select from the live catalog or enter another Fireworks model ID. The default endpoint is https://api.fireworks.ai/inference/v1; configure a different endpoint through model.base_url in config.yaml, not .env.

NovitaAI

NovitaAI is the AI-native cloud for builders and agents. Its three product lines are Model API for 200+ models, Agent Sandbox for building and running AI agents, and GPU Cloud for scalable compute, all available from one platform.

# Use any available model
hermes chat --provider novita --model moonshotai/kimi-k2.5
# Requires: NOVITA_API_KEY in ~/.hermes/.env

# Short alias
hermes chat --provider novita-ai --model deepseek/deepseek-v3-0324

Or set it permanently in config.yaml:

model:
  provider: "novita"
  default: "moonshotai/kimi-k2.5"
  base_url: "https://api.novita.ai/openai/v1"

Get your API key at novita.ai/settings/key-management. The base URL can be overridden with NOVITA_BASE_URL.

z.ai / GLM

# Requires: GLM_API_KEY in ~/.hermes/.env
hermes chat --provider zai --model glm-5

Z.AI Endpoint Auto-Detection: When using the Z.AI / GLM provider, Hermes automatically probes multiple endpoints (global, China, coding variants) to find one that accepts your API key. You don't need to set GLM_BASE_URL manually, the working endpoint is detected and cached automatically.

Kimi / Moonshot AI

International (api.moonshot.ai):

# Requires: KIMI_API_KEY in ~/.hermes/.env
hermes chat --provider kimi-coding --model kimi-for-coding

China (api.moonshot.cn):

# Requires: KIMI_CN_API_KEY in ~/.hermes/.env
hermes chat --provider kimi-coding-cn --model kimi-k2.5

MiniMax

Global endpoint:

# Requires: MINIMAX_API_KEY in ~/.hermes/.env
hermes chat --provider minimax --model MiniMax-M2.7

China endpoint:

# Requires: MINIMAX_CN_API_KEY in ~/.hermes/.env
hermes chat --provider minimax-cn --model MiniMax-M2.7

Qwen Cloud / DashScope

# Requires: DASHSCOPE_API_KEY in ~/.hermes/.env
hermes chat --provider alibaba --model qwen3.5-plus

Alibaba Cloud (Coding Plan)

If you're subscribed to Alibaba's Coding Plan (a pricing SKU separate from standard DashScope API access), Hermes exposes it as its own first-class provider: alibaba-coding-plan. Endpoint: https://coding-intl.dashscope.aliyuncs.com/v1. It's OpenAI-compatible like the regular alibaba provider but with a different base URL and billing surface.

model:
  provider: alibaba_coding  # alias for alibaba-coding-plan
  model: qwen3-coder-plus

Or from the CLI:

hermes chat --provider alibaba_coding --model qwen3-coder-plus

alibaba_coding uses the same DASHSCOPE_API_KEY your alibaba entry already uses, no separate key needed, just a different routing target. Before this provider was registered, users who set provider: alibaba_coding in config.yaml silently fell through to OpenRouter routing.

xAI (Grok), Responses API + Prompt Caching

xAI is wired through the Responses API (codex_responses transport) for automatic reasoning support on Grok 4 models, no reasoning_effort parameter needed, the server reasons by default.

# Requires: XAI_API_KEY in ~/.hermes/.env
hermes chat --provider xai --model grok-4-fast-reasoning

SuperGrok and X Premium+ subscribers can sign in with browser OAuth instead of using an API key, pick xAI Grok OAuth (SuperGrok / Premium+) in hermes model, or run hermes auth add xai-oauth. The same OAuth bearer token is automatically reused by direct-to-xAI tools (TTS, image gen, video gen, transcription). See the xAI Grok OAuth guide for the full flow, and if Hermes runs on a remote host, also see OAuth over SSH / Remote Hosts for the required ssh -L tunnel.

When using xAI as a provider (any base URL containing x.ai), Hermes automatically enables prompt caching by sending the x-grok-conv-id header with every API request. This routes requests to the same server within a conversation session, allowing xAI's infrastructure to reuse cached system prompts and conversation history. No configuration is needed, caching activates automatically when an xAI endpoint is detected and a session ID is available. This reduces latency and cost for multi-turn conversations.

xAI also ships a dedicated TTS endpoint (/v1/tts). Select xAI TTS in hermes tools → Voice & TTS, or see the Voice & TTS page for config.

Retired xAI model migration (May 15, 2026): xAI is retiring grok-4*, grok-3, grok-code-fast-1, and grok-imagine-image-pro on 2026-05-15. hermes doctor and hermes chat startup both detect any config still pointing at a retired ref and print the recommended replacement. Use hermes migrate xai for a one-shot config rewrite, dry-run by default, add --apply to write changes (a timestamped config.yaml.bak-pre-migrate-xai-* backup is created automatically).

hermes migrate xai          # preview replacements
hermes migrate xai --apply  # rewrite ~/.hermes/config.yaml in place

xAI Web Search backend: When the Web Search toolset is enabled, web.backend: xai routes search through xAI's hosted search endpoint using the same XAI_API_KEY / OAuth credentials. No additional setup required if xAI is already configured as a provider.

Ollama Cloud

Ollama Cloud hosts the same open-weight catalog as local Ollama but without the GPU requirement. Pick it in hermes model as Ollama Cloud, paste your API key from ollama.com/settings/keys, and Hermes auto-discovers the available models.

hermes model
# → pick "Ollama Cloud"
# → paste your OLLAMA_API_KEY
# → select from discovered models (gpt-oss:120b, glm-4.6:cloud, qwen3-coder:480b-cloud, etc.)

Or config.yaml directly:

model:
  provider: "ollama-cloud"
  default: "gpt-oss:120b"

The model catalog is fetched dynamically from ollama.com/v1/models and cached for one hour. model:tag notation (e.g. qwen3-coder:480b-cloud) is preserved through normalization, don't use dashes.

Ollama Cloud vs local Ollama: Both speak the same OpenAI-compatible API. Cloud is a first-class provider (--provider ollama-cloud, OLLAMA_API_KEY); local Ollama is reached via the Custom Endpoint flow (base URL http://localhost:11434/v1, no key). Use cloud for large models you can't run locally; use local for privacy or offline work.

AWS Bedrock

Anthropic Claude, Amazon Nova, DeepSeek v3.2, Meta Llama 4, and other models via AWS Bedrock. Uses the AWS SDK (boto3) credential chain, no API key, just standard AWS auth.

# Simplest, named profile in ~/.aws/credentials
hermes chat --provider bedrock --model us.anthropic.claude-sonnet-4-6

# Or with explicit env vars
AWS_PROFILE=myprofile AWS_REGION=us-east-1 hermes chat --provider bedrock --model us.anthropic.claude-sonnet-4-6

Or permanently in config.yaml:

model:
  provider: "bedrock"
  default: "us.anthropic.claude-sonnet-4-6"
  bedrock:
    region: "us-east-1"  # or set AWS_REGION
    # profile: "myprofile"  # or set AWS_PROFILE
    # discovery: true  # auto-discover region from IAM
    # guardrail:
    #   guardrail_identifier: "your-guardrail-id"
    #   guardrail_version: "DRAFT"

Authentication uses the standard boto3 chain: explicit AWS_ACCESS_KEY_ID/AWS_SECRET_ACCESS_KEY, AWS_PROFILE from ~/.aws/credentials, IAM role on EC2/ECS/Lambda, IMDS, or SSO. No env var is required if you're already authenticated with the AWS CLI.

Bedrock uses the Converse API under the hood, requests are translated to Bedrock's model-agnostic shape, so the same config works for Claude, Nova, DeepSeek, and Llama models. Set BEDROCK_BASE_URL only if you're calling a non-default regional endpoint.

Google Vertex AI

Gemini models on Google Cloud Vertex AI via Vertex's OpenAI-compatible endpoint. Authentication is OAuth2, a short-lived access token (~1 hour) minted from a service-account JSON or Application Default Credentials (ADC). There is no static API key; Hermes mints and auto-refreshes the token for you, including re-minting on a mid-session 401.

# Service account JSON (recommended for servers / gateways)
echo "VERTEX_CREDENTIALS_PATH=/path/to/service-account.json" >> ~/.hermes/.env

# or Application Default Credentials
gcloud auth application-default login

hermes model
# → "Google Vertex AI" → project → region → model

Or in config.yaml (project/region are non-secret and live here; the credential path stays in .env):

model:
  provider: "vertex"
  default: "google/gemini-3-flash-preview"  # Vertex requires the google/ prefix
  vertex:
    project_id: "my-gcp-project"  # blank → use the project embedded in the credentials
    region: "global"  # required for the Gemini 3.x previews

VERTEX_PROJECT_ID / VERTEX_REGION env vars override the config.yaml values. Hermes lazy-installs google-auth on first use; run hermes setup if the managed install needs repair.

Qwen Portal (OAuth)

Alibaba's Qwen Portal with browser-based OAuth login. Pick Qwen OAuth (Portal) in hermes model, sign in through the browser, and Hermes persists the refresh token.

hermes model
# → pick "Qwen OAuth (Portal)"
# → browser opens; sign in with your Alibaba account
# → confirm, credentials are saved to ~/.hermes/auth.json

hermes chat  # uses portal.qwen.ai/v1 endpoint

Or configure config.yaml:

model:
  provider: "qwen-oauth"
  default: "qwen3-coder-plus"

Set HERMES_QWEN_BASE_URL only if the portal endpoint relocates (default: https://portal.qwen.ai/v1).

Qwen OAuth vs Qwen Cloud (Alibaba DashScope): qwen-oauth uses the consumer-facing Qwen Portal with OAuth login, ideal for individual users. The alibaba provider uses Qwen Cloud (Alibaba DashScope) with a DASHSCOPE_API_KEY, ideal for programmatic / production workloads. Both route to Qwen-family models but live at different endpoints.

MiniMax (OAuth)

MiniMax-M2.7 via browser OAuth login, no API key needed. Pick MiniMax (OAuth) in hermes model, sign in through the browser, and Hermes persists the access + refresh tokens. Uses the Anthropic Messages-compatible endpoint (/anthropic) under the hood.

hermes model
# → pick "MiniMax (OAuth)"
# → browser opens; sign in with your MiniMax account (global or CN region)
# → confirm, credentials are saved to ~/.hermes/auth.json

hermes chat  # uses api.minimax.io/anthropic endpoint

Or configure config.yaml:

model:
  provider: "minimax-oauth"
  default: "MiniMax-M2.7"

Set HERMES_MINIMAX_BASE_URL only if the endpoint relocates (default: https://api.minimax.io/anthropic).

Other API-Key Providers

These providers follow the same pattern: set the environment variable in ~/.hermes/.env, then use --provider with the provider ID.

Provider	Env Variable	Provider ID	Aliases
Arcee AI	ARCEEAI_API_KEY	arcee	arcee-ai, arceeai
GMI Cloud	GMI_API_KEY	gmi	gmi-cloud, gmicloud
Xiaomi MiMo	XIAOMI_API_KEY	xiaomi	mimo, xiaomi-mimo
Tencent TokenHub	TOKENHUB_API_KEY	tencent-tokenhub	tencent, tokenhub, tencentmaas
OpenCode Zen	OPENCODE_ZEN_API_KEY	opencode-zen
OpenCode Go	OPENCODE_GO_API_KEY	opencode-go
DeepSeek	DEEPSEEK_API_KEY	deepseek
Hugging Face	HF_TOKEN	huggingface	hf
Google / Gemini	GOOGLE_API_KEY or GEMINI_API_KEY	gemini
NVIDIA Build	NVIDIA_API_KEY	nvidia
StepFun	STEPFUN_API_KEY	stepfun
Kilo Code	KILOCODE_API_KEY	kilocode

Example usage:

# Arcee AI (Trinity models)
hermes chat --provider arcee --model trinity-large-thinking

# GMI Cloud, use the exact model ID returned by GMI's /v1/models endpoint
hermes chat --provider gmi --model zai-org/GLM-5.1-FP8

# Xiaomi MiMo
hermes chat --provider xiaomi --model mimo-v2-pro

# Tencent TokenHub (Hy3 Preview)
hermes chat --provider tencent-tokenhub --model hy3-preview

Custom Endpoint

For any OpenAI-compatible endpoint not covered by a first-class provider, use the Custom Endpoint flow in hermes model. This saves the configuration in config.yaml.

hermes model
# → choose "Custom endpoint"
# → enter base URL (e.g., http://localhost:11434/v1 for local Ollama)
# → optionally enter API key

Azure AI Foundry

hermes model
# → "Azure AI Foundry" (provider: `azure-foundry`)
# → uses Azure OpenAI / Foundry endpoint and key

LM Studio

hermes model
# → "LM Studio" (provider: `lmstudio`, optional `LM_API_KEY`)

Permanent Configuration in config.yaml

All provider and model settings can be stored permanently in ~/.hermes/config.yaml. The file uses YAML format. Here is a comprehensive example:

model:
  provider: "openrouter"
  default: "openai/gpt-4o"
  # Optional: override base URL
  base_url: "https://openrouter.ai/api/v1"

  # Model key alias: both `default:` and `model:` work identically
  # model: "openai/gpt-4o"

  # Provider-specific sections
  bedrock:
    region: "us-east-1"
    # profile: "myprofile"
    # discovery: true
    # guardrail:
    #   guardrail_identifier: "your-guardrail-id"
    #   guardrail_version: "DRAFT"

  vertex:
    project_id: "my-gcp-project"
    region: "global"