Hermes Agent Browser Automation: Setup, Tools, and Configuration

This page documents the browser automation integration in Hermes Agent, covering all supported backends: Browserbase, Browser Use, Firecrawl, Camofox, local Chromium-family CDP, and the local agent-browser CLI. Once configured, an agent can navigate websites, interact with page elements, fill forms, extract information, take screenshots, run JavaScript, and use the Chrome DevTools Protocol directly.

What It Does

Hermes Agent includes a full browser automation toolset with multiple backend options:

• Browserbase cloud mode via Browserbase for managed cloud browsers and anti-bot tooling
• Browser Use cloud mode via Browser Use as an alternative cloud browser provider
• Firecrawl cloud mode via Firecrawl for cloud browsers with built-in scraping
• Camofox local mode via Camofox for local anti-detection browsing (Firefox-based fingerprint spoofing)
• Local Chromium-family CDP, connect browser tools to your own Chrome, Brave, Chromium, or Edge instance using /browser connect
• Local browser mode via the agent-browser CLI and a local Chromium installation

In all modes, the agent can navigate websites, interact with page elements, fill forms, and extract information.

Pages are represented as accessibility trees (text-based snapshots), making them ideal for LLM agents. Interactive elements get ref IDs (like @e1, @e2) that the agent uses for clicking and typing.

Key capabilities:

• Multi-provider cloud execution, Browserbase, Browser Use, or Firecrawl, no local browser needed
• Local Chromium-family integration, attach to your running Chrome, Brave, Chromium, or Edge browser via CDP for hands-on browsing
• Built-in stealth, random fingerprints, CAPTCHA solving, residential proxies (Browserbase)
• Session isolation, each task gets its own browser session
• Automatic cleanup, inactive sessions are closed after a timeout
• Vision analysis, screenshot + AI analysis for visual understanding

Setup

Nous Subscribers

If you have a paid Nous Portal subscription, you can use browser automation through the Tool Gateway without any separate API keys. New installs can run hermes setup --portal to log in and turn on every gateway tool at once; existing installs can pick Nous Subscription as the browser provider via hermes model or hermes tools.

Browserbase cloud mode

To use Browserbase-managed cloud browsers, add:

# Add to ~/.hermes/.env
BROWSERBASE_API_KEY = ***
BROWSERBASE_PROJECT_ID = your-project-id-here

Get your credentials at browserbase.com.

Browser Use cloud mode

To use Browser Use as your cloud browser provider, add:

# Add to ~/.hermes/.env
BROWSER_USE_API_KEY = ***

Get your API key at browser-use.com. Browser Use provides a cloud browser via its REST API. If both Browserbase and Browser Use credentials are set, Browserbase takes priority.

Firecrawl cloud mode

To use Firecrawl as your cloud browser provider, add:

# Add to ~/.hermes/.env
FIRECRAWL_API_KEY = fc-***

Get your API key at firecrawl.dev. Then select Firecrawl as your browser provider:

hermes setup tools
# → Browser Automation → Firecrawl

Optional settings:

# Self-hosted Firecrawl instance (default: https://api.firecrawl.dev)
FIRECRAWL_API_URL = http://localhost:3002

# Session TTL in seconds (default: 300)
FIRECRAWL_BROWSER_TTL = 600

Hybrid routing: cloud for public URLs, local for LAN/localhost

When a cloud provider is configured, Hermes auto-spawns a local Chromium sidecar for URLs that resolve to a private/loopback/LAN address (localhost, 127.0.0.1, 192.168.x.x, 10.x.x.x, 172.16-31.x.x, *.local, *.lan, *.internal, IPv6 loopback ::1, link-local 169.254.x.x). Public URLs continue to use the cloud provider in the same conversation.

This solves the common "I'm developing locally but using Browserbase" workflow, the agent can screenshot your dashboard at http://localhost:3000 AND scrape https://github.com without you switching providers or disabling the SSRF guard. The cloud provider never sees the private URL.

The feature is on by default. To disable it (all URLs go to the configured cloud provider, as before):

# ~/.hermes/config.yaml
browser:
  cloud_provider: browserbase
  auto_local_for_private_urls: false

With auto-routing disabled, private URLs are rejected with "Blocked: URL targets a private or internal address" unless you also set browser.allow_private_urls: true (which lets the cloud provider attempt them, usually won't work since Browserbase etc. can't reach your LAN).

Requirements: the local sidecar uses the same agent-browser CLI as pure local mode, so you need it installed (hermes setup tools → Browser Automation auto-installs it). Post-navigation redirects from a public URL onto a private address are still blocked (you can't use a redirect-to-internal trick to reach your LAN through the public path).

Camofox local mode

Camofox is a self-hosted Node.js server wrapping Camoufox (a Firefox fork with C++ fingerprint spoofing). It provides local anti-detection browsing without cloud dependencies.

# Clone the Camofox browser server first
git clone https://github.com/jo-inc/camofox-browser
cd camofox-browser

# Build and start with Docker using the default container settings
# (auto-detects arch: aarch64 on M1/M2, x86_64 on Intel)
make up

# Stop and remove the default container
make down

# Force a clean rebuild (for example, after upgrading VERSION/RELEASE)
make reset

# Just download binaries without building
make fetch

# Override arch or version explicitly
make up ARCH=x86_64
make up VERSION=135.0.1 RELEASE=beta.24

make up starts the default container immediately. If you want custom runtime settings such as a larger Node heap, VNC, or a persistent profile directory, build the image first and then run it yourself:

# Build the image without starting the default container
make build

# Start with persistence, VNC live view, and a larger Node heap
mkdir -p ~/.camofox-docker
docker run -d \
  --name camofox-browser \
  --restart unless-stopped \
  -p 9377:9377 \
  -p 6080:6080 \
  -p 5901:5900 \
  -e CAMOFOX_PORT=9377 \
  -e ENABLE_VNC=1 \
  -e VNC_BIND=0.0.0.0 \
  -e VNC_RESOLUTION=1920x1080 \
  -e MAX_OLD_SPACE_SIZE=2048 \
  -v ~/.camofox-docker:/root/.camofox \
  camofox-browser:135.0.1-aarch64

With VNC enabled, the browser runs in headed mode and can be watched live in your browser at http://localhost:6080 (noVNC). You can also connect a native VNC client to localhost:5901. If you already ran make up, stop and remove that default container before starting the custom one:

make down
# then run the custom docker run command above

Then set in ~/.hermes/.env:

CAMOFOX_URL=http://localhost:9377

If Camofox is running in Docker and you want it to open web apps served from the host machine, enable loopback rewriting. CAMOFOX_URL should still point at the host-published control API, but page URLs such as http://127.0.0.1:3000 must be opened from inside the container as http://host.docker.internal:3000:

# ~/.hermes/config.yaml
browser:
  camofox:
    rewrite_loopback_urls: true
    loopback_host_alias: host.docker.internal  # default; use a LAN IP if needed

Equivalent env vars:

CAMOFOX_REWRITE_LOOPBACK_URLS=true
CAMOFOX_LOOPBACK_HOST_ALIAS=host.docker.internal

The rewrite only applies to page navigation URLs with loopback hosts (localhost, 127.0.0.1, ::1). It does not change CAMOFOX_URL. Leave it disabled for non-Docker Camofox installs, where the browser already runs on the host and loopback URLs are correct. Or configure via hermes tools → Browser Automation → Camofox.

When CAMOFOX_URL is set, all browser tools automatically route through Camofox instead of Browserbase or agent-browser.

Persistent browser sessions

By default, each Camofox session gets a random identity, cookies and logins don't survive across agent restarts. To enable persistent browser sessions, add the following to ~/.hermes/config.yaml:

browser:
  camofox:
    managed_persistence: true

Then fully restart Hermes so the new config is picked up.

Nested path matters: Hermes reads browser.camofox.managed_persistence, not a top-level managed_persistence. A common mistake is writing:

# ❌ Wrong, Hermes ignores this
managed_persistence: true

If the flag is placed at the wrong path, Hermes silently falls back to a random ephemeral userId and your login state will be lost on every session.

What Hermes does

• Sends a deterministic profile-scoped userId to Camofox so the server can reuse the same Firefox profile across sessions.
• Skips server-side context destruction on cleanup, so cookies and logins survive between agent tasks.
• Scopes the userId to the active Hermes profile, so different Hermes profiles get different browser profiles (profile isolation).

What Hermes does not do

• It does not force persistence on the Camofox server. Hermes only sends a stable userId; the server must honor it by mapping that userId to a persistent Firefox profile directory.
• If your Camofox server build treats every request as ephemeral (e.g. always calls browser.newContext() without loading a stored profile), Hermes cannot make those sessions persist. Make sure you are running a Camofox build that implements userId-based profile persistence.

Verify it's working

• Start Hermes and your Camofox server.
• Open Google (or any login site) in a browser task and sign in manually.
• End the browser task normally.
• Start a new browser task.
• Open the same site again, you should still be signed in.

If step 5 logs you out, the Camofox server isn't honoring the stable userId. Double-check your config path, confirm you fully restarted Hermes after editing config.yaml, and verify your Camofox server version supports persistent per-user profiles.

Where state lives

Hermes derives the stable userId from the profile-scoped directory ~/.hermes/browser_auth/camofox/ (or the equivalent under $HERMES_HOME for non-default profiles). The actual browser profile data lives on the Camofox server side, keyed by that userId. To fully reset a persistent profile, clear it on the Camofox server and remove the corresponding Hermes profile's state directory.

Externally managed Camofox sessions

When another app drives the visible Camofox browser (a desktop assistant, a custom integration, another agent), configure Hermes to operate inside that same identity instead of spawning its own isolated profile. Three knobs control the behavior:

Setting	Env var	Effect
browser.camofox.user_id	CAMOFOX_USER_ID	Camofox userId Hermes uses when creating tabs. Setting this opts the session into "externally managed" mode.
browser.camofox.session_key	CAMOFOX_SESSION_KEY	sessionKey (a.k.a. listItemId) sent on tab creation. Used to match an existing tab during adoption. Defaults to a per-task value if unset.
browser.camofox.adopt_existing_tab	CAMOFOX_ADOPT_EXISTING_TAB	When true, Hermes calls GET /tabs?userId= on first use and reuses an existing tab before creating a new one.

Env vars take precedence over config.yaml. Either form works:

browser:
  camofox:
    user_id: shared-camofox
    session_key: visible-tab
    adopt_existing_tab: true

CAMOFOX_USER_ID=shared-camofox
CAMOFOX_SESSION_KEY=visible-tab
CAMOFOX_ADOPT_EXISTING_TAB=true

What changes when user_id is set:

• Hermes skips destructive cleanup at task end (same as managed_persistence: true). The other app's tab/cookies/profile survive.
• Hermes does not call DELETE /sessions/, that endpoint wipes all user data, so it would nuke the external app's session if it fired.

How tab adoption works (when adopt_existing_tab: true):

• On the first browser tool call after a process start, Hermes issues GET /tabs?userId= (5-second timeout).
• If any tab in the response has listItemId == session_key, Hermes adopts the most recently created one in that group.
• Otherwise, Hermes adopts the most recently created tab for the user (any listItemId).
• If no tabs exist or the request fails, Hermes falls back to creating a new tab on the next operation.

Adoption only fires until tab_id is populated for the session. If the external app closes the adopted tab mid-run, the next browser tool call will surface a Camofox error, Hermes does not re-poll for a fresh tab on every call.

Picking session_key: if you want Hermes to reliably attach to a specific existing tab, set session_key to the listItemId the external app used when creating it. If you leave session_key unset and only set user_id, Hermes generates a per-task session_key (task_), Hermes will share cookies and the profile with the external app, but will open its own tab alongside instead of reusing one.

Concurrency note: the external app and Hermes can drive the same Camofox userId simultaneously, but Camofox does not coordinate per-tab focus between clients. Coordinate ownership at the application layer (e.g. the external app pauses while Hermes runs).

VNC live view

When Camofox runs in headed mode (with a visible browser window), it exposes a VNC port in its health check response. Hermes automatically discovers this and includes the VNC URL in navigation responses, so the agent can share a link for you to watch the browser live.

Local Chromium-family browser via CDP (/browser connect)

Instead of a cloud provider, you can attach Hermes browser tools to your own running Chrome, Brave, Chromium, or Edge instance via the Chrome DevTools Protocol (CDP). This is useful when you want to see what the agent is doing in real-time, interact with pages that require your own cookies/sessions, or avoid cloud browser costs.

Note: /browser connect is an interactive-CLI slash command, it is not dispatched by the gateway. If you try to run it inside a WebUI, Telegram, Discord, or other gateway chat, the message will be sent to the agent as plain text and the command will not execute. Start Hermes from the terminal (hermes or hermes chat) and issue /browser connect there.

In the CLI, use:

/browser connect
# Auto-launch/connect to a local Chromium-family browser at http://127.0.0.1:9222

/browser connect ws://host:port
# Connect to a specific CDP endpoint

/browser status
# Check current connection

/browser disconnect
# Detach and return to cloud/local mode

If a browser isn't already running with remote debugging, Hermes will attempt to auto-launch a supported Chromium-family browser with --remote-debugging-port=9222. Detection includes Brave, Google Chrome, Chromium, and Microsoft Edge, with common Linux install paths such as /opt/brave-bin/brave and /snap/bin/brave.

To start a Chromium-family browser manually with CDP enabled, use a dedicated user-data-dir so the debug port actually comes up even if the browser is already running with your normal profile:

# Linux, Brave
brave-browser \
  --remote-debugging-port=9222 \
  --user-data-dir=$HOME/.hermes/chrome-debug \
  --no-first-run \
  --no-default-browser-check &

# Linux, Google Chrome
google-chrome \
  --remote-debugging-port=9222 \
  --user-data-dir=$HOME/.hermes/chrome-debug \
  --no-first-run \
  --no-default-browser-check &

# macOS, Brave
"/Applications/Brave Browser.app/Contents/MacOS/Brave Browser" \
  --remote-debugging-port=9222 \
  --user-data-dir="$HOME/.hermes/chrome-debug" \
  --no-first-run \
  --no-default-browser-check &

# macOS, Google Chrome
"/Applications/Google Chrome.app/Contents/MacOS/Google Chrome" \
  --remote-debugging-port=9222 \
  --user-data-dir="$HOME/.hermes/chrome-debug" \
  --no-first-run \
  --no-default-browser-check &

Then launch the Hermes CLI and run /browser connect.

Why --user-data-dir? Without it, launching a Chromium-family browser while a regular instance is already running typically opens a new window on the existing process, and that existing process was not started with --remote-debugging-port, so port 9222 never opens. A dedicated user-data-dir forces a fresh browser process where the debug port actually listens. --no-first-run --no-default-browser-check skips the first-launch wizard for the fresh profile.

When connected via CDP, all browser tools (browser_navigate, browser_click, etc.) operate on your live browser instance instead of spinning up a cloud session.

WSL2 + Windows Chrome: prefer MCP over /browser connect

If Hermes runs inside WSL2 but the Chrome window you want to control runs on the Windows host, /browser connect is often not the best path. Why:

• /browser connect expects Hermes itself to reach a usable CDP endpoint
• modern Chrome live-debugging sessions often expose a host-local endpoint that is not directly reachable from WSL the same way a classic 9222 port is
• even when Windows Chrome is debuggable, the cleanest integration is often to let a Windows-side browser MCP server attach to Chrome and let Hermes talk to that MCP server

For that setup, prefer chrome-devtools-mcp through Hermes MCP support. See the MCP guide for the practical setup.

Local browser mode

If you do not set any cloud credentials and don't use /browser connect, Hermes can still use the browser tools through a local Chromium install driven by agent-browser.

Optional Environment Variables

# Residential proxies for better CAPTCHA solving (default: "true")
BROWSERBASE_PROXIES=true

# Advanced stealth with custom Chromium, requires Scale Plan (default: "false")
BROWSERBASE_ADVANCED_STEALTH=false

# Session reconnection after disconnects, requires paid plan (default: "true")
BROWSERBASE_KEEP_ALIVE=true

# Custom session timeout in seconds (max 21600 = 6 hours) (default: project default)
# Examples: 600 (10min), 1800 (30min), 21600 (6h max)
BROWSERBASE_SESSION_TIMEOUT=1800

# Inactivity timeout before auto-cleanup in seconds (default: 120)
BROWSER_INACTIVITY_TIMEOUT=120

# Extra Chromium launch flags (comma- or newline-separated). Hermes auto-injects
# `--no-sandbox,--disable-dev-shm-usage` when it detects root or AppArmor-restricted
# unprivileged user namespaces (Ubuntu 23.10+, DGX Spark, many container images),
# so most users don't need to set this. Set it manually only if you need a flag
# Hermes doesn't add automatically; setting it disables the auto-injection.
AGENT_BROWSER_ARGS=--no-sandbox

Install agent-browser CLI

npm install -g agent-browser
# Or install locally in the repo:
npm install

The browser toolset must be included in your config's toolsets list or enabled via hermes config set toolsets '["hermes-cli", "browser"]'.