Hermes Agent Browser Automation: Setup, Tools, and Configuration

{ "title": "Hermes Agent Browser Automation Reference", "body": "## Overview\n\nHermes Agent provides a comprehensive browser automation toolset for LLM agents. It enables navigation, interaction with dynamic web pages, form filling, and information extraction through multiple backend options. The toolset supports cloud-managed browsers with anti-bot protection, local browser integration via CDP, and local anti-detection browsing.\n\n## When to Use Browser Automation\n\nUse the browser toolset when you need to interact with dynamic web pages that require clicking buttons, filling forms, or handling dynamic content, rather than simple information retrieval. Choose cloud-managed browsers (Browserbase, Browser Use, Firecrawl) for anti-bot protection. Use local Chromium-family integration via CDP for real-time visibility or to leverage existing cookies and sessions. Use Camofox for local anti-detection browsing with Firefox-based fingerprint spoofing. Use the agent-browser CLI for simple local Chromium installations.\n\n## Prerequisites\n\nBefore using the browser automation toolset, ensure the following prerequisites are met:\n\n- Hermes Agent is installed and configured.\n- For cloud modes, valid API keys from the respective providers (Browserbase, Browser Use, Firecrawl) are required.\n- For Camofox, Docker must be installed and the Camofox server must be cloned and running.\n- For local Chromium CDP, a Chromium-family browser (Chrome, Brave, Chromium, Edge) must be installed.\n- For local browser mode, the agent-browser CLI must be installed globally (npm install -g agent-browser) or locally.\n- The browser toolset must be included in the configuration's toolsets list or enabled via hermes config set toolsets '[\"hermes-cli\", \"browser\"]'.\n- For hybrid routing, the agent-browser CLI must be installed (auto-installed via hermes setup tools -> Browser Automation).\n- For persistent Camofox sessions, the Camofox server build must implement userId-based profile persistence.\n- For externally managed Camofox sessions, another application must be driving the visible Camofox browser.\n- For the browser_cdp tool, a CDP endpoint must be reachable at session start (via /browser connect or browser.cdp_url set).\n- For dialog handling on Browserbase, the CDP supervisor must be active (automatic for Browserbase sessions).\n- For headed mode, local browser mode must be used (not cloud).\n- For session recording, browser.record_sessions: true must be set in the configuration.\n- For WSL2 with Windows Chrome, the chrome-devtools-mcp MCP server must be set up on the Windows side.\n\n## Capabilities\n\nThe browser automation toolset provides the following capabilities:\n\n- Multi-provider cloud execution: Browserbase, Browser Use, or Firecrawl.\n- Local Chromium-family integration via CDP: Chrome, Brave, Chromium, Edge.\n- Built-in stealth: Random fingerprints, CAPTCHA solving, residential proxies (Browserbase).\n- Session isolation: Each task gets its own browser session.\n- Automatic cleanup: Inactive sessions are closed after a timeout.\n- Vision analysis: Screenshot plus AI analysis for visual understanding.\n- Pages represented as accessibility trees: Text-based snapshots with ref IDs for interactive elements.\n- Hybrid routing: Cloud for public URLs, local sidecar for private or LAN URLs (auto by default).\n- Persistent browser sessions (Camofox): Deterministic profile-scoped userId.\n- Externally managed Camofox sessions: Operate inside an existing identity or tab.\n- VNC live view: Camofox headed mode.\n- Session recording: WebM video files.\n- Headed mode: Visible Chromium window kept open between turns.\n- Browser console log retrieval and JavaScript evaluation.\n- Raw CDP passthrough: browser_cdp for operations not covered by other tools.\n- Native JS dialog handling: alert, confirm, prompt, beforeunload with configurable policy.\n- Frame tree in snapshot: Including cross-origin OOPIF iframes.\n\n## Configuration\n\n### Cloud Provider Parameters\n\n| Parameter | Meaning | Required |\n|-----------|---------|----------|\n| BROWSERBASE_API_KEY | API key for Browserbase cloud mode | Required for Browserbase cloud mode |\n| BROWSERBASE_PROJECT_ID | Project ID for Browserbase cloud mode | Required for Browserbase cloud mode |\n| BROWSER_USE_API_KEY | API key for Browser Use cloud mode | Required for Browser Use cloud mode |\n| FIRECRAWL_API_KEY | API key for Firecrawl cloud mode (prefix fc-) | Required for Firecrawl cloud mode |\n| FIRECRAWL_API_URL | Self-hosted Firecrawl instance URL | Optional; default https://api.firecrawl.dev |\n| FIRECRAWL_BROWSER_TTL | Session TTL in seconds for Firecrawl | Optional; default 300 |\n\n### Camofox Parameters\n\n| Parameter | Meaning | Required |\n|-----------|---------|----------|\n| CAMOFOX_URL | URL of Camofox server (e.g., http://localhost:9377) | Required for Camofox mode |\n| CAMOFOX_REWRITE_LOOPBACK_URLS | Enable loopback URL rewriting for Docker Camofox | Optional; default false |\n| CAMOFOX_LOOPBACK_HOST_ALIAS | Host alias for loopback rewriting (default host.docker.internal) | Optional |\n| CAMOFOX_USER_ID | Camofox userId for externally managed sessions | Optional; setting opts into externally managed mode |\n| CAMOFOX_SESSION_KEY | sessionKey (listItemId) for matching existing tab | Optional; defaults to per-task value |\n| CAMOFOX_ADOPT_EXISTING_TAB | When true, reuse existing tab before creating new one | Optional; default false |\n| browser.camofox.managed_persistence | Enable persistent Camofox sessions (deterministic userId) | Optional; default false |\n| browser.camofox.user_id | Camofox userId for externally managed sessions (config.yaml) | Optional |\n| browser.camofox.session_key | sessionKey for matching existing tab (config.yaml) | Optional |\n| browser.camofox.adopt_existing_tab | Reuse existing tab (config.yaml) | Optional |\n\n### Browser Behavior Parameters\n\n| Parameter | Meaning | Required |\n|-----------|---------|----------|\n| browser.cloud_provider | Cloud provider name (e.g., browserbase) for hybrid routing config | Required when configuring hybrid routing |\n| browser.auto_local_for_private_urls | Enable hybrid routing: cloud for public, local sidecar for private URLs | Optional; default true |\n| browser.allow_private_urls | Allow cloud provider to attempt private URLs (usually fails) | Optional; default false |\n| BROWSERBASE_PROXIES | Enable residential proxies for Browserbase | Optional; default true |\n| BROWSERBASE_ADVANCED_STEALTH | Enable advanced stealth with custom Chromium (requires Scale Plan) | Optional; default false |\n| BROWSERBASE_KEEP_ALIVE | Enable session reconnection after disconnects (requires paid plan) | Optional; default true |\n| BROWSERBASE_SESSION_TIMEOUT | Custom session timeout in seconds (max 21600) | Optional; default project default |\n| BROWSER_INACTIVITY_TIMEOUT | Inactivity timeout before auto-cleanup in seconds | Optional; default 120 |\n| AGENT_BROWSER_ARGS | Extra Chromium launch flags (comma/newline-separated). Setting disables auto-injection | Optional |\n| AGENT_BROWSER_HEADED | Enable headed mode (visible window) | Optional; default false |\n| browser.record_sessions | Enable session recording as WebM | Optional; default false |\n| browser.headed | Enable headed mode (config.yaml) | Optional; default false |\n| browser.dialog_policy | Policy for native JS dialogs: must_respond (default), auto_dismiss, auto_accept | Optional |\n| browser.dialog_timeout_s | Safety timeout for must_respond policy (default 300s) | Optional |\n| browser.restrict_evaluate | Enable strict denylist over sensitive JS primitives in browser_console evaluation | Optional; default false |\n| browser.cdp_url | CDP endpoint URL for persistent connection (config.yaml) | Optional; enables browser_cdp tool |\n\n## Setup Procedures\n\n### Setup Browserbase Cloud Mode\n\n1. Add BROWSERBASE_API_KEY and BROWSERBASE_PROJECT_ID to ~/.hermes/.env.\n2. Get credentials at browserbase.com.\n\n### Setup Browser Use Cloud Mode\n\n1. Add BROWSER_USE_API_KEY to ~/.hermes/.env.\n2. Get API key at browser-use.com.\n3. Note: If both Browserbase and Browser Use credentials are set, Browserbase takes priority.\n\n### Setup Firecrawl Cloud Mode\n\n1. Add FIRECRAWL_API_KEY (fc-***) to ~/.hermes/.env.\n2. Get API key at firecrawl.dev.\n3. Select Firecrawl as browser provider via hermes setup tools -> Browser Automation -> Firecrawl.\n4. Optional: Set FIRECRAWL_API_URL (default https://api.firecrawl.dev) and FIRECRAWL_BROWSER_TTL (default 300s) in .env.\n\n### Setup Camofox Local Mode\n\n1. Clone Camofox server:\n \n git clone https://github.com/jo-inc/camofox-browser && cd camofox-browser\n \n2. Build and start with Docker: make up (auto-detects arch).\n3. Stop or remove default container: make down.\n4. Force clean rebuild: make reset.\n5. Download binaries only: make fetch.\n6. Override arch or version: make up ARCH=x86_64 VERSION=135.0.1 RELEASE=beta.24.\n7. For custom runtime (larger heap, VNC, persistent profile): make build, then run a custom docker run command with -p 9377:9377, -p 6080:6080, -p 5901:5900, environment variables CAMOFOX_PORT=9377, ENABLE_VNC=1, VNC_BIND=0.0.0.0, VNC_RESOLUTION=1920x1080, MAX_OLD_SPACE_SIZE=2048, and volume mount ~/.camofox-docker:/root/.camofox.\n8. Stop the default container with make down before starting the custom one.\n9. Set CAMOFOX_URL=http://localhost:9377 in ~/.hermes/.env.\n10. Optional: Enable loopback rewriting in config.yaml (browser.camofox.rewrite_loopback_urls: true, browser.camofox.loopback_host_alias: host.docker.internal) or environment variables CAMOFOX_REWRITE_LOOPBACK_URLS=true, CAMOFOX_LOOPBACK_HOST_ALIAS=host.docker.internal.\n11. Or configure via hermes tools -> Browser Automation -> Camofox.\n12. When CAMOFOX_URL is set, all browser tools route through Camofox.\n\n### Enable Persistent Camofox Sessions\n\n1. Add browser.camofox.managed_persistence: true to ~/.hermes/config.yaml (nested under browser.camofox, not top-level).\n2. Fully restart Hermes to pick up the configuration.\n\n### Setup Externally Managed Camofox Sessions\n\n1. Set browser.camofox.user_id (or CAMOFOX_USER_ID environment variable) to the desired userId.\n2. Optionally set browser.camofox.session_key (or CAMOFOX_SESSION_KEY) to match an existing tab's listItemId.\n3. Optionally set browser.camofox.adopt_existing_tab: true (or CAMOFOX_ADOPT_EXISTING_TAB=true) to reuse an existing tab.\n4. Environment variables take precedence over config.yaml.\n\n### Connect to Local Chromium-Family Browser via CDP (/browser connect)\n\n1. Ensure Hermes is running in the CLI (not WebUI, Telegram, or Discord).\n2. Issue /browser connect to auto-launch or connect to a browser at http://127.0.0.1:9222.\n3. Or issue /browser connect ws://host:port for a specific CDP endpoint.\n4. Use /browser status to check the connection.\n5. Use /browser disconnect to detach and return to cloud or local mode.\n6. If the browser is not running with remote debugging, Hermes auto-launches a supported Chromium-family browser with --remote-debugging-port=9222 (detects Brave, Chrome, Chromium, Edge).\n7. For manual launch, use a dedicated user-data-dir and flags:\n \n --remote-debugging-port=9222 --user-data-dir=$HOME/.hermes/chrome-debug --no-first-run --no-default-browser-check\n \n8. Then run /browser connect in the Hermes CLI.\n\n### Use Browser Tools in a Conversation\n\n1. Call browser_navigate(url) first to initialize the session.\n2. Call browser_snapshot() to get the accessibility tree with ref IDs.\n3. Use browser_click(ref), browser_type(ref, text), browser_scroll(direction), browser_press(key), browser_back(), browser_get_images(), browser_vision(prompt), browser_console(clear=True, expression), browser_cdp(method, params, target_id, frame_id), browser_dialog(action, prompt_text) as needed.\n4. For dialog handling: call browser_snapshot to see pending_dialogs, then browser_dialog(action='accept'/'dismiss', prompt_text='...'), then re-snapshot.\n\n### Enable Session Recording\n\n1. Set browser.record_sessions: true in config.yaml (default false).\n2. Recording starts automatically on the first browser_navigate.\n3. Saves to ~/.hermes/browser_recordings/ when the session closes.\n4. Recordings older than 72 hours are auto-cleaned.\n\n### Enable Headed Mode\n\n1. Set browser.headed: true in config.yaml (default false) or environment variable AGENT_BROWSER_HEADED=1.\n2. Chromium launches with a visible window; the window stays open between turns.\n3. Per-turn cleanup is skipped; idle sessions are reaped after browser.inactivity_timeout (default 120s).\n4. Only affects local browser, not cloud sessions.\n\n## Tool Reference\n\n### browser_navigate\n\nInitializes a browser session and navigates to the specified URL. This must be the first tool call in any browser workflow. The URL must be a fully qualified HTTP or HTTPS address. If the session does not exist, it is created automatically. Returns the page title and URL after navigation.\n\n### browser_snapshot\n\nReturns the current page as an accessibility tree. Each interactive element has a ref ID that other tools use for targeting. The snapshot includes a frame_tree field showing the document hierarchy, including cross-origin OOPIF iframes. The frame_tree is capped at 30 frames and OOPIF depth 2. If limits are hit, a truncated: true flag is set. The snapshot also includes a pending_dialogs field when native JS dialogs (alert, confirm, prompt, beforeunload) are active. Large pages may be truncated or LLM-summarized at 15000 characters. The complete snapshot is saved to ~/.hermes/cache/web/ for read_file paging.\n\n### browser_click\n\nClicks an element identified by its ref ID from the snapshot. The ref must be a string value from a previous browser_snapshot call. Returns success or an error if the ref is invalid or the element is not interactable.\n\n### browser_type\n\nTypes text into an input element identified by its ref ID. The text is typed character by character to simulate human input. Use this for form fields, search boxes, and text areas. Returns success or an error if the ref is invalid.\n\n### browser_scroll\n\nScrolls the page in the specified direction. Valid directions are up, down, left, right. The scroll amount is approximately one viewport height or width. Returns the new scroll position.\n\n### browser_press\n\nPresses a keyboard key. Valid keys include Enter, Escape, Tab, ArrowUp, ArrowDown, ArrowLeft, ArrowRight, Backspace, Delete, Home, End, PageUp, PageDown. Returns success or an error if the key is invalid.\n\n### browser_back\n\nNavigates to the previous page in the browser history. Returns the new page title and URL. If there is no previous page, returns an error.\n\n### browser_get_images\n\nReturns a list of image URLs found on the current page. Each image entry includes the URL and alt text. Use this to identify images for visual analysis.\n\n### browser_vision\n\nTakes a screenshot of the current page and sends it to an AI model for visual analysis. The prompt parameter describes what to look for. Returns the AI analysis result. This is useful for understanding visual layout, identifying images, or verifying page state.\n\n### browser_console\n\nRetrieves browser console logs or evaluates JavaScript expressions. When called with clear=True, it clears the console log and returns the previous log entries. When called with expression, it evaluates the JavaScript expression in the page context and returns the result. The expression must be a string of JavaScript code. By default, evaluation is unrestricted. When browser.restrict_evaluate: true is set, a denylist blocks sensitive primitives like fetch, cookie, localStorage, sessionStorage, indexedDB, openDatabase, Worker, SharedWorker, ServiceWorker, WebSocket, EventSource, XMLHttpRequest, ActiveXObject, netscape, java, Packages, sun, getClass, javaClass, javaSystem, javaNet, javaIo, javaLang, javaUtil, javaMath, javaText, javaBeans, javaRmi, javaSecurity, javaSql, javaAwt, javaSwing, javaApplet, javaScript, javaDebug, javaManagement, javaNaming, javaSpi, javaTime, javaUtilConcurrent, javaUtilFunction, javaUtilRegex, javaUtilStream, javaUtilZip, javaUtilJar, javaUtilLogging, javaUtilProperties, javaUtilResourceBundle, javaUtilScanner, javaUtilFormatter, javaUtilCurrency, javaUtilLocale, javaUtilUUID, javaUtilOptional, javaUtilComparator, javaUtilIterator, javaUtilList, javaUtilMap, javaUtilSet, javaUtilQueue, javaUtilDeque, javaUtilCollection, javaUtilArrays, javaUtilCollections, javaUtilObjects, javaUtilSpliterator, javaUtilSplittableRandom, javaUtilConcurrentAtomic, javaUtilConcurrentLocks, javaUtilConcurrentForkJoin, javaUtilConcurrentThreadPool, javaUtilConcurrentExecutors, javaUtilConcurrentScheduled, javaUtilConcurrentTimeUnit, javaUtilConcurrentCountDownLatch, javaUtilConcurrentCyclicBarrier, javaUtilConcurrentSemaphore, javaUtilConcurrentExchanger, javaUtilConcurrentPhaser, javaUtilConcurrentCompletionService, javaUtilConcurrentExecutorCompletionService, javaUtilConcurrentFuture, javaUtilConcurrentCallable, javaUtilConcurrentRunnableFuture, javaUtilConcurrentRunnableScheduledFuture, javaUtilConcurrentScheduledFuture, javaUtilConcurrentDelayed, javaUtilConcurrentBlockingQueue, javaUtilConcurrentLinkedBlockingQueue, javaUtilConcurrentArrayBlockingQueue, javaUtilConcurrentPriorityBlockingQueue, javaUtilConcurrentSynchronousQueue, javaUtilConcurrentDelayQueue, javaUtilConcurrentTransferQueue, javaUtilConcurrentLinkedTransferQueue, javaUtilConcurrentConcurrentMap, javaUtilConcurrentConcurrentHashMap, javaUtilConcurrentConcurrentNavigableMap, javaUtilConcurrentConcurrentSkipListMap, javaUtilConcurrentConcurrentSkipListSet, javaUtilConcurrentCopyOnWriteArrayList, javaUtilConcurrentCopyOnWriteArraySet, javaUtilConcurrentThreadLocalRandom, javaUtilConcurrentForkJoinPool, javaUtilConcurrentForkJoinTask, javaUtilConcurrentRecursiveAction, javaUtilConcurrentRecursiveTask, javaUtilConcurrentCountedCompleter, javaUtilConcurrentCompletableFuture, javaUtilConcurrentCompletionStage, javaUtilConcurrentFlow, javaUtilConcurrentSubmissionPublisher, javaUtilConcurrentRejectedExecutionHandler, javaUtilConcurrentThreadFactory, javaUtilConcurrentUncaughtExceptionHandler, javaUtilConcurrentExecutor, javaUtilConcurrentExecutorService, javaUtilConcurrentScheduledExecutorService, javaUtilConcurrentCallable, javaUtilConcurrentFuture, javaUtilConcurrentRunnableFuture, javaUtilConcurrentRunnableScheduledFuture, javaUtilConcurrentScheduledFuture, javaUtilConcurrentDelayed, javaUtilConcurrentBlockingQueue, javaUtilConcurrentLinkedBlockingQueue, javaUtilConcurrentArrayBlockingQueue, javaUtilConcurrentPriorityBlockingQueue, javaUtilConcurrentSynchronousQueue, javaUtilConcurrentDelayQueue, javaUtilConcurrentTransferQueue, javaUtilConcurrentLinkedTransferQueue, javaUtilConcurrentConcurrentMap, javaUtilConcurrentConcurrentHashMap, javaUtilConcurrentConcurrentNavigableMap, javaUtilConcurrentConcurrentSkipListMap, javaUtilConcurrentConcurrentSkipListSet, javaUtilConcurrentCopyOnWriteArrayList, javaUtilConcurrentCopyOnWriteArraySet, javaUtilConcurrentThreadLocalRandom, javaUtilConcurrentForkJoinPool, javaUtilConcurrentForkJoinTask, javaUtilConcurrentRecursiveAction, javaUtilConcurrentRecursiveTask, javaUtilConcurrentCountedCompleter, javaUtilConcurrentCompletableFuture, javaUtilConcurrentCompletionStage, javaUtilConcurrentFlow, javaUtilConcurrentSubmissionPublisher, javaUtilConcurrentRejectedExecutionHandler, javaUtilConcurrentThreadFactory, javaUtilConcurrentUncaughtExceptionHandler, javaUtilConcurrentExecutor, javaUtilConcurrentExecutorService, javaUtilConcurrentScheduledExecutorService. This denylist can block legitimate expressions containing these words.\n\n### browser_cdp\n\nSends a raw CDP command to the browser. This tool is only available when a CDP endpoint is reachable at session start (via /browser connect or browser.cdp_url set). It is not available in the default local agent-browser, Camofox, or cloud providers currently. Calls are stateless; sessions do not persist between calls. Parameters include method (string, required), params (object, optional), target_id (string, optional), frame_id (string, optional). When frame_id is provided, the call is routed through the supervisor's live session to the specific frame. This is useful for cross-origin OOPIF iframes where is_oopif=true in the snapshot frame_tree. Returns the raw CDP response.\n\n### browser_dialog\n\nHandles a native JS dialog (alert, confirm, prompt, beforeunload). The action parameter must be accept or dismiss. The prompt_text parameter is optional and only used for prompt dialogs. Before calling this tool, call browser_snapshot to see pending_dialogs. After handling, re-snapshot to confirm the dialog is cleared. The dialog policy is configured via browser.dialog_policy. The default policy is must_respond, which requires the agent to explicitly handle dialogs. If the agent does not respond within browser.dialog_timeout_s (default 300s), a safety auto-dismiss fires. The auto_dismiss policy automatically dismisses dialogs. The auto_accept policy automatically accepts dialogs. Dialog detection uses a persistent CDP supervisor (one WebSocket per task). It is not available on Camofox or the default local agent-browser. On Browserbase, dialog handling uses an injected XHR bridge (not Page.handleJavaScriptDialog) because Browserbase auto-dismisses native dialogs server-side.\n\n## Session Management\n\n### Session Lifecycle\n\nEach browser task gets its own isolated session. Sessions are created on the first browser_navigate call. Sessions are automatically cleaned up when the conversation ends or after BROWSER_INACTIVITY_TIMEOUT (default 120 seconds) of inactivity. Cloud sessions expire based on the provider's plan settings. For Browserbase, BROWSERBASE_SESSION_TIMEOUT can be set to a custom value (max 21600 seconds). For Firecrawl, FIRECRAWL_BROWSER_TTL controls session TTL (default 300 seconds).\n\n### Session Recording\n\nWhen browser.record_sessions: true is set in config.yaml, recording starts automatically on the first browser_navigate. Recordings are saved as WebM video files to ~/.hermes/browser_recordings/ when the session closes. Recordings older than 72 hours are auto-cleaned.\n\n### Headed Mode\n\nWhen browser.headed: true is set in config.yaml or AGENT_BROWSER_HEADED=1 environment variable, Chromium launches with a visible window. The window stays open between turns. Per-turn cleanup is skipped. Idle sessions are reaped after browser.inactivity_timeout (default 120 seconds). Headed mode only affects the local browser, not cloud sessions.\n\n### Hybrid Routing\n\nWhen browser.auto_local_for_private_urls: true (default), the system automatically routes public URLs through the cloud provider and private or LAN URLs through a local sidecar. This avoids SSRF guard blocks for private addresses. Post-navigation redirects from a public URL to a private address are still blocked. When browser.allow_private_urls: true is set, the cloud provider attempts private URLs (which usually fail). The hybrid routing feature requires the agent-browser CLI to be installed.\n\n## Cache and Cleanup\n\n### Screenshot Cache\n\nScreenshots taken during browser sessions are stored in ~/.hermes/cache/screenshots/. These are auto-cleaned after 24 hours.\n\n### Snapshot Cache\n\nComplete snapshots (when truncated) are saved to ~/.hermes/cache/web/ for read_file paging. This allows the agent to access the full accessibility tree even when the snapshot is summarized.\n\n### Recording Cleanup\n\nSession recordings older than 72 hours are auto-cleaned from ~/.hermes/browser_recordings/.\n\n## Provider-Specific Details\n\n### Browserbase\n\nBrowserbase provides cloud-managed browsers with built-in stealth features. It supports residential proxies (BROWSERBASE_PROXIES, default true), advanced stealth with custom Chromium (BROWSERBASE_ADVANCED_STEALTH, requires Scale Plan), and session reconnection (BROWSERBASE_KEEP_ALIVE, requires paid plan). If paid features are not available on the plan, Hermes auto-falls back (first keepAlive, then proxies). Dialog handling on Browserbase uses an injected XHR bridge because Browserbase auto-dismisses native dialogs server-side.\n\n### Browser Use\n\nBrowser Use provides cloud-managed browsers. If both Browserbase and Browser Use credentials are set, Browserbase takes priority.\n\n### Firecrawl\n\nFirecrawl provides cloud-managed browsers. API keys must have the prefix fc-. A self-hosted instance can be specified via FIRECRAWL_API_URL. Session TTL is controlled by FIRECRAWL_BROWSER_TTL (default 300 seconds).\n\n### Camofox\n\nCamofox provides local anti-detection browsing with Firefox-based fingerprint spoofing. It runs as a Docker container. The server URL is configured via CAMOFOX_URL. When CAMOFOX_URL is set, all browser tools route through Camofox. Camofox supports VNC live view (ports 6080 for web VNC, 5901 for VNC). Persistent sessions require browser.camofox.managed_persistence: true and a Camofox server that implements userId-based profile persistence. Externally managed sessions allow another application to drive the visible Camofox browser while Hermes operates inside it.\n\n### Local Chromium CDP\n\nLocal Chromium-family browsers (Chrome, Brave, Chromium, Edge) can be connected via CDP using /browser connect. This provides real-time visibility and access to existing cookies and sessions. The browser_cdp tool is available when a CDP endpoint is reachable.\n\n### Agent-Browser CLI\n\nThe agent-browser CLI provides simple local Chromium installations. It is installed globally via npm install -g agent-browser or locally. It is required for hybrid routing and local browser mode.\n\n## Constraints and Caveats\n\n- Text-based interaction: The toolset relies on the accessibility tree, not pixel coordinates.\n- Snapshot size: Large pages may be truncated or LLM-summarized at 15000 characters; the complete snapshot is saved to ~/.hermes/cache/web/ for read_file paging.\n- Session timeout: Cloud sessions expire based on the provider's plan settings.\n- Cost: Cloud sessions consume provider credits; sessions are auto-cleaned when the conversation ends or after inactivity. Use /browser connect for free local browsing.\n- No file downloads: The browser cannot download files.\n- /browser connect is an interactive-CLI slash command only; it does not work in WebUI, Telegram, Discord, or other gateway chat.\n- The browser_cdp tool is only available when a CDP endpoint is reachable at session start (via /browser connect or browser.cdp_url set). It is not available in the default local agent-browser, Camofox, or cloud providers currently.\n- browser_cdp calls are stateless; sessions do not persist between calls.\n- browser_console evaluation is unrestricted by default; restrict_evaluate: true applies a denylist that matches primitive names, which can block legitimate expressions containing words like fetch or cookie.\n- Hybrid routing: Post-navigation redirects from a public URL to a private address are still blocked.\n- Camofox managed_persistence: Hermes sends a stable userId but the server must honor it; if the server treats every request as ephemeral, persistence will not work.\n- Camofox externally managed: Adoption only fires until tab_id is populated; if the external application closes the adopted tab mid-run, the next tool call surfaces an error (no re-poll).\n- Camofox concurrency: The external application and Hermes can drive the same userId simultaneously, but Camofox does not coordinate per-tab focus; coordinate ownership at the application layer.\n- Headed mode only affects the local browser; cloud sessions (Browserbase) are unaffected.\n- If paid Browserbase features are not available on the plan, Hermes auto-falls back (first keepAlive, then proxies).\n- WSL2 + Windows Chrome: Prefer MCP (chrome-devtools-mcp) over /browser connect.\n- The snapshot frame_tree is capped to 30 frames and OOPIF depth 2; a truncated: true flag surfaces when limits are hit.\n- Dialog detection uses a persistent CDP supervisor (one WebSocket per task). It is not available on Camofox or the default local agent-browser.\n- On Browserbase, dialog handling uses an injected XHR bridge (not Page.handleJavaScriptDialog) because Browserbase auto-dismisses native dialogs server-side.\n- browser.camofox.managed_persistence must be nested under browser.camofox; a top-level managed_persistence is ignored silently.\n- Setting AGENT_BROWSER_ARGS disables auto-injection of --no-sandbox,--disable-dev-shm-usage.\n- Screenshots stored in ~/.hermes/cache/screenshots/ are auto-cleaned after 24 hours.\n- Session recordings older than 72 hours are auto-cleaned.\n\n## Failure Modes\n\n- If managed_persistence is placed at the wrong path (top-level instead of browser.camofox.managed_persistence), Hermes silently falls back to a random ephemeral userId and login state is lost on every session.\n- If the Camofox server does not implement userId-based profile persistence, persistent sessions will not work (the verification step logs you out).\n- If an external application closes an adopted tab mid-run, the next browser tool call surfaces a Camofox error (Hermes does not re-poll).\n- If a cloud provider cannot reach private or LAN URLs, they fail unless auto_local_for_private_urls is enabled.\n- If both Browserbase and Browser Use credentials are set, Browserbase takes priority (Browser Use is ignored).\n- If /browser connect is used in a non-CLI chat, the command is not executed (sent as plain text).\n- If the browser is not started with --remote-debugging-port and a dedicated user-data-dir, port 9222 may not open (existing process reused).\n- If the SSRF guard blocks private URLs and auto_local_for_private_urls is disabled, private URLs are rejected with "Blocked: URL targets a private or internal address".\n- If browser_console evaluation hits restricted primitive names with restrict_evaluate: true, legitimate expressions may be blocked.\n- If the snapshot exceeds 15000 characters, it is truncated or summarized; the agent must use read_file to see the full tree.\n- If the dialog policy is must_respond and the agent does not respond within dialog_timeout_s, a safety auto-dismiss fires.\n- If frame tree limits are hit (30 frames, OOPIF depth 2), a truncated: true flag is set; the agent needs browser_cdp for the full tree.\n\n## Example Workflows\n\n### Filling Out a Web Form\n\n1. Navigate to the signup page.\n2. Take a snapshot to see the form fields.\n3. Type the email and password into the identified refs.\n4. Click the create account button.\n5. Take another snapshot to confirm success.\n\n### Researching Dynamic Content\n\n1. Navigate to GitHub trending.\n2. Take a snapshot with full=true to read the repo list.\n3. Return the formatted results.\n\n### Browser Console Evaluation\n\nUse browser_console(expression=\"document.querySelector('h1').textContent\") to get the page title, or browser_console(expression=\"JSON.stringify(performance.timing)\") for performance data.\n\n### CDP Patterns\n\n- List tabs: browser_cdp(method=\"Target.getTargets\")\n- Handle dialog: browser_cdp(method=\"Page.handleJavaScriptDialog\", params={\"accept\": true})\n- Evaluate JS in a tab: browser_cdp(method=\"Runtime.evaluate\", params={\"expression\": \"document.title\", \"returnByValue\": true}, target_id=\"<tabId>\")\n- Get cookies: browser_cdp(method=\"Network.getAllCookies\")\n\n### Cross-Origin Iframe CDP\n\nPass frame_id from browser_snapshot.frame_tree.children[] where is_oopif=true to route the CDP call through the supervisor's live session.\n\n### Dialog Workflow\n\n1. Call browser_snapshot to see pending_dialogs.\n2. Call browser_dialog(action='accept' or 'dismiss', prompt_text='...').\n3. Re-snapshot to confirm the dialog is cleared.\n\n### Camofox Persistence Verification\n\n1. Start Hermes and Camofox.\n2. Open Google and sign in.\n3. End the task.\n4. Start a new task.\n5. Open the same site. You should still be signed in. If logged out, the Camofox server is not honoring the stable userId.\n\n## Troubleshooting\n\n### Connection Issues\n\n- If /browser connect fails, ensure the browser is running with --remote-debugging-port=9222 and a dedicated user-data-dir.\n- If cloud provider credentials are rejected, verify the API key format and project ID.\n- If Camofox connection fails, ensure the Docker container is running and CAMOFOX_URL is correct.\n\n### Session Issues\n\n- If sessions are not persisting, check that browser.camofox.managed_persistence is nested under browser.camofox (not top-level).\n- If sessions are timing out too quickly, adjust BROWSER_INACTIVITY_TIMEOUT or BROWSERBASE_SESSION_TIMEOUT.\n- If sessions are not cleaning up, check that the conversation has ended or the inactivity timeout has passed.\n\n### Dialog Issues\n\n- If dialogs are not detected, ensure the CDP supervisor is active (automatic for Browserbase sessions, not available on Camofox or default local agent-browser).\n- If dialogs are auto-dismissed unexpectedly, check the browser.dialog_policy setting.\n- If browser_dialog fails, call browser_snapshot first to see pending_dialogs.\n\n### Snapshot Issues\n\n- If the snapshot is truncated, use read_file to access the full tree from ~/.hermes/cache/web/.\n- If the snapshot is empty, ensure the page has loaded and is not blocked.\n- If frame tree limits are hit, use browser_cdp for the full tree.\n\n### Console Evaluation Issues\n\n- If browser_console evaluation is blocked, check if browser.restrict_evaluate: true is set and the expression contains denylisted words.\n- If console logs are not returned, ensure clear=True is set to retrieve previous logs.\n\n### CDP Issues\n\n- If browser_cdp is not available, ensure a CDP endpoint is reachable via /browser connect or browser.cdp_url.\n- If CDP calls fail, verify the method name and parameters are valid for the target browser version.\n\n### Camofox Issues\n\n- If persistent sessions do not work, verify the Camofox server implements userId-based profile persistence.\n- If externally managed sessions fail, ensure the external application is running and the userId matches.\n- If loopback URLs are not rewriting, enable CAMOFOX_REWRITE_LOOPBACK_URLS and set CAMOFOX_LOOPBACK_HOST_ALIAS correctly.\n\n### Hybrid Routing Issues\n\n- If private URLs fail, ensure browser.auto_local_for_private_urls: true is set.\n- If public URLs are routed locally, check that the cloud provider credentials are valid.\n- If post-navigation redirects to private addresses fail, this is expected behavior.\n\n## Security Considerations\n\n### SSRF Protection\n\nThe system includes SSRF guards that block private or internal addresses when using cloud providers. The browser.auto_local_for_private_urls feature routes private URLs through a local sidecar to avoid these blocks. When browser.allow_private_urls is false (default), private URLs are rejected with "Blocked: URL targets a private or internal address".\n\n### JavaScript Evaluation Security\n\nBy default, browser_console evaluation is unrestricted. When browser.restrict_evaluate: true is set, a denylist blocks sensitive JavaScript primitives. This can block legitimate expressions containing words like fetch or cookie. Use this setting when you need to prevent the agent from accessing sensitive browser APIs.\n\n### Session Isolation\n\nEach task gets its own browser session. Sessions are automatically cleaned up when the conversation ends or after inactivity. This prevents data leakage between tasks.\n\n### Camofox Identity Management\n\nCamofox uses Firefox-based fingerprint spoofing for anti-detection. Persistent sessions use a deterministic userId for profile scoping. Externally managed sessions allow another application to control the identity while Hermes operates inside it. Coordinate ownership at the application layer to avoid conflicts.\n\n## Performance Considerations\n\n### Snapshot Size\n\nLarge pages may be truncated or LLM-summarized at 15000 characters. The complete snapshot is saved to ~/.hermes/cache/web/ for read_file paging. Use read_file to access the full tree when the snapshot is summarized.\n\n### Frame Tree Limits\n\nThe snapshot frame_tree is capped to 30 frames and OOPIF depth 2. A truncated: true flag surfaces when limits are hit. Use browser_cdp for the full tree when needed.\n\n### Session Cleanup\n\nInactive sessions are closed after BROWSER_INACTIVITY_TIMEOUT (default 120 seconds). Cloud sessions expire based on the provider's plan settings. Screenshots are auto-cleaned after 24 hours. Session recordings are auto-cleaned after 72 hours.\n\n### Headed Mode Overhead\n\nHeaded mode keeps the Chromium window open between turns, which consumes more resources. Per-turn cleanup is skipped. Idle sessions are reaped after browser.inactivity_timeout (default 120 seconds).\n\n## Compatibility\n\n### Supported Browsers\n\n- Chromium-family browsers: Chrome, Brave, Chromium, Edge (for CDP and local mode)\n- Firefox (for Camofox anti-detection)\n\n### Supported Platforms\n\n- Linux (primary)\n- macOS\n- Windows (via WSL2 with chrome-devtools-mcp MCP server)\n\n### Supported Cloud Providers\n\n- Browserbase\n- Browser Use\n- Firecrawl\n\n### Unsupported Features\n\n- File downloads\n- Pixel-based interaction (uses accessibility tree)\n- Cloud sessions in headed mode\n- /browser connect in non-CLI gateways\n- browser_cdp in default local agent-browser, Camofox, or cloud providers\n\n## API Reference\n\n### Tool Signatures\n\n\nbrowser_navigate(url: string) -> {title: string, url: string}\nbrowser_snapshot() -> {tree: object, frame_tree: object, pending_dialogs: array, truncated: boolean}\nbrowser_click(ref: string) -> {success: boolean}\nbrowser_type(ref: string, text: string) -> {success: boolean}\nbrowser_scroll(direction: string) -> {position: number}\nbrowser_press(key: string) -> {success: boolean}\nbrowser_back() -> {title: string, url: string}\nbrowser_get_images() -> {images: array}\nbrowser_vision(prompt: string) -> {analysis: string}\nbrowser_console(clear: boolean, expression: string) -> {logs: array, result: any}\nbrowser_cdp(method: string, params: object, target_id: string, frame_id: string) -> {result: object}\nbrowser_dialog(action: string, prompt_text: string) -> {success: boolean}\n\n\n### Error Responses\n\nAll tools return errors in a consistent format:\n\n{error: string, details: string}\n\n\nCommon error messages include:\n- "Session not initialized. Call browser_navigate first."\n- "Invalid ref ID: {ref}"\n- "Element not interactable: {ref}"\n- "Blocked: URL targets a private or internal address"\n- "Dialog not found. Call browser_snapshot to see pending dialogs."\n- "CDP endpoint not reachable."\n- "Method not found: {method}"\n- "Camofox server error: {details}"\n- "Cloud provider error: {details}"" }