Watchers: Poll RSS, JSON APIs, and GitHub with Watermark Dedup

Watchers are lightweight scripts that poll external sources on a schedule and only surface new items. They are a good fit when you want to monitor an RSS feed, track GitHub issues or releases, or watch a JSON endpoint for fresh entries, and you need the agent to report only what changed since the last check. The scripts handle deduplication through a watermark file, so the first run records a baseline and subsequent runs emit only new content.

What it does

Each watcher script follows the same pattern: fetch data from the source, compare the returned IDs against a stored watermark of previously seen IDs, update the watermark, and print new items to stdout. If nothing is new, stdout is empty. The agent interprets empty output as "nothing to report" and stays silent. This keeps notifications clean and avoids repeating the same items.

Three ready-made scripts are included, plus a shared watermark helper. You can run them ad-hoc from the terminal or wire them into a cron job. The agent can also schedule the cron job for you through a natural language prompt.

Before you start

The skill is optional and must be installed on demand. Once installed, the scripts live at $HERMES_HOME/skills/devops/watchers/scripts/. The environment variable WATCHER_STATE_DIR controls where state files are written; it defaults to $HERMES_HOME/watcher-state/. That directory must be writable by the agent's runtime. If you are using a Docker or Modal backend, arbitrary host paths may not be visible, so stick with the default location.

For GitHub watchers, set GITHUB_TOKEN in ${HERMES_HOME:-~/.hermes}/.env to avoid the anonymous rate limit of 60 requests per hour. Without a token, the script still works but may hit the limit quickly if you poll frequently.

Ready-made scripts

All three scripts accept a --name argument that keys the state file. The state file is written as $HERMES_HOME/watcher-state/<name>.json.

Script	What it watches	Dedup key
watch_rss.py	RSS 2.0 or Atom feed URL	/
watch_http_json.py	Any JSON endpoint returning a list of objects	Configurable id field
watch_github.py	GitHub issues / pulls / releases / commits for a repo	id / sha

All three share these behaviours:

• First run records a baseline and never replays existing feed items.
• Watermark is a bounded ID set with a maximum of 500 entries to cap memory.
• Output format: ## <title>\n\n<content>\n\n<link> per item.
• Empty stdout on no new items.
• Non-zero exit on fetch errors.

Usage

Run a watcher directly from the terminal tool. The examples below assume the skill is installed and the scripts are in the expected path.

Watch an RSS feed:

python $HERMES_HOME/skills/devops/watchers/scripts/watch_rss.py \
  --name hn --url https://news.ycombinator.com/rss --max 5

Watch a GitHub repo (set GITHUB_TOKEN in ${HERMES_HOME:-~/.hermes}/.env to avoid the 60 req/hr anonymous rate limit):

python $HERMES_HOME/skills/devops/watchers/scripts/watch_github.py \
  --name hermes-issues --repo NousResearch/hermes-agent --scope issues

Poll an arbitrary JSON API:

python $HERMES_HOME/skills/devops/watchers/scripts/watch_http_json.py \
  --name api --url https://api.example.com/events \
  --id-field event_id --items-path data.events

Wiring into cron

You can ask the agent to schedule a cron job with a prompt like:

Every 15 minutes, run watch_rss.py --name hn --url https://news.ycombinator.com/rss. If it prints anything, summarize the headlines and deliver them. If it prints nothing, stay silent.

The agent invokes the script via the terminal tool inside the cron job's agent loop; no changes to cron's built-in --script flag are needed.

State files

Every watcher writes $HERMES_HOME/watcher-state/<name>.json. Inspect the state file for a feed named hn:

cat $HERMES_HOME/watcher-state/hn.json

Force a replay so the next run is treated as the first poll:

rm $HERMES_HOME/watcher-state/hn.json

Writing your own

All three scripts use the same template: load watermark, fetch, diff, save, emit. The file scripts/_watermark.py is the shared helper; import it to get atomic writes, a bounded ID set, and first-run baseline handling for free. See any of the three reference scripts for how little boilerplate it takes.

When not to use it

If you need real-time streaming or webhook-based delivery, a polling watcher is the wrong tool. The scripts are designed for periodic checks on a cron schedule. They also assume the external source returns a list of items with stable IDs; if the source does not provide unique identifiers, deduplication will not work correctly.

Limits and gotchas

1. Printing a "no new items" header every tick. Callers rely on empty stdout = silent. If you print anything on an empty delta, you spam the channel. The shipped scripts handle this; custom scripts must too.
2. Expecting the first run to emit items. It won't, first run records a baseline. If you need an initial digest, delete the state file after the first run or add a --prime-with-latest N flag in your own script.
3. Unbounded watermark growth. The shared helper caps at 500 IDs. Raise it for high-churn feeds; lower it on constrained filesystems.
4. Putting the state dir where the agent's sandbox can't write. $HERMES_HOME/watcher-state/ is always writable. Docker/Modal backends may not see arbitrary host paths.

What pairs with this

The watchers skill is part of the DevOps category. It works naturally with the agent's terminal tool for execution and with cron for scheduling. The same state directory convention is used across other Hermes Agent skills that persist local data.