Mantis Slack Desktop QA Runbook

Mantis Slack desktop QA serves as the real user interface testing path for Slack related bugs that demand a Linux desktop environment, VNC rescue access, Slack Web, an actual OpenClaw gateway, screenshot and video capture, plus a PR evidence comment. Choose this lane when unit tests or the headless Slack live lane cannot demonstrate the bug.

Storage model

Mantis relies on three storage tiers:

Provider image - belongs to Crabbox, kept in the cloud provider's account. Contains machine capabilities (Chrome/Chromium, ffmpeg, scrot, Node/corepack/pnpm, native build tools) along with empty cache directories.
Warm lease state - belongs to the active operator session. May store a logged-in browser profile, /var/cache/crabbox/pnpm, and a ready source checkout while the lease remains active.
Mantis artifacts - belongs to the OpenClaw execution. Located under .artifacts/qa-e2e/mantis/...; GitHub Actions uploads these and the Mantis GitHub App posts inline evidence as a comment on the PR.

Never embed secrets, browser cookies, Slack login state, repository checkouts, node_modules, or dist/ inside a provider image.

GitHub dispatch

Trigger the workflow from main:

gh workflow run mantis-slack-desktop-smoke.yml \
  --ref main \
  -f candidate_ref=<trusted-ref-or-sha> \
  -f pr_number=<pr-number> \
  -f scenario_id=slack-canary \
  -f crabbox_provider=aws \
  -f keep_vm=false \
  -f hydrate_mode=source

Access to candidate_ref is restricted since the workflow uses live credentials: it must point to the current main lineage, a release tag, or an open PR head within openclaw/openclaw.

The workflow generates:

uploaded artifact mantis-slack-desktop-smoke-<run-id>-<attempt>
inline PR comment from the Mantis GitHub App
slack-desktop-smoke.png, slack-desktop-smoke.mp4
slack-desktop-smoke-preview.gif, slack-desktop-smoke-change.mp4
mantis-slack-desktop-smoke-summary.json, mantis-slack-desktop-smoke-report.md
remote logs: slack-desktop-command.log, openclaw-gateway.log, chrome.log, ffmpeg.log

The PR comment gets updated in place using the hidden  marker.

Local CLI

Cold source proof:

pnpm openclaw qa mantis slack-desktop-smoke \
  --provider aws \
  --class standard \
  --gateway-setup \
  --credential-source convex \
  --credential-role maintainer \
  --provider-mode live-frontier \
  --model openai/gpt-5.4 \
  --alt-model openai/gpt-5.4 \
  --scenario slack-canary \
  --hydrate-mode source

Keep the VM available for VNC rescue:

pnpm openclaw qa mantis slack-desktop-smoke \
  --provider aws \
  --class standard \
  --gateway-setup \
  --scenario slack-canary \
  --keep-lease

Open VNC:

crabbox vnc --provider aws --id <cbx_id> --open

Reuse a warm lease:

pnpm openclaw qa mantis slack-desktop-smoke \
  --provider aws \
  --lease-id <cbx_id-or-slug> \
  --gateway-setup \
  --scenario slack-canary \
  --hydrate-mode source

Only use --hydrate-mode prehydrated when the reused remote workspace already contains node_modules and a compiled dist/; Mantis fails closed otherwise.

Prove native Slack approval UI:

pnpm openclaw qa mantis slack-desktop-smoke \
  --provider aws \
  --class standard \
  --approval-checkpoints \
  --credential-source convex \
  --credential-role maintainer \
  --hydrate-mode source

--approval-checkpoints and --gateway-setup are mutually exclusive. It runs the opt-in slack-approval-exec-native and slack-approval-plugin-native scenarios unless you provide an explicit approval-checkpoint --scenario; other Slack scenarios get rejected before the VM starts. The Slack QA runner writes each checkpoint JSON file from the real Slack API message it observed, then the remote watcher renders that message into approval-checkpoints/<scenario>-pending.png and approval-checkpoints/<scenario>-resolved.png. The run fails if any checkpoint JSON, message evidence, ack JSON, or rendered screenshot is missing or empty.

Cold GitHub Actions leases lack Slack Web cookies, so their browser capture may land on the Slack sign-in screen. For approval-checkpoint proof, rely on the rendered checkpoint images and Slack QA artifacts instead of slack-desktop-smoke.png. Only use a kept warm lease with a manually logged-in Slack Web profile when the browser screenshot itself must display Slack Web.

Hydrate modes

Mode	Use when	Remote behavior	Tradeoff
`source`	Normal PR proof, cold machines, CI	Runs `pnpm install --frozen-lockfile --prefer-offline` and `pnpm build` inside the VM	Slowest, strongest source-checkout proof
`prehydrated`	You intentionally prepared a reused lease	Requires existing `node_modules` and `dist/`; skips install/build	Fast, but only valid for operator-controlled warm leases

GitHub Actions always prepares the candidate checkout before the VM run. Its pnpm store gets cached by OS, Node version, and lockfile. The VM source run also reuses /var/cache/crabbox/pnpm when available.

Timing interpretation

mantis-slack-desktop-smoke-report.md contains phase timings:

crabbox.warmup - cloud provider boot, desktop/browser readiness, SSH.
crabbox.inspect - lease metadata lookup.
credentials.prepare - Convex credential lease acquisition.
crabbox.remote_run - sync, browser launch, OpenClaw install/build or hydrate validation, gateway startup, screenshot, and video capture.
artifacts.copy - rsync back from the VM.

crabbox.remote_run may display accepted when Crabbox returns a non-zero remote status but Mantis copied metadata proving either the OpenClaw gateway setup completed or the Slack QA command itself exited successfully. Treat accepted as pass-with-explanation, not a failed scenario.

If a run is slow:

When warmup dominates, either prebake the Crabbox provider image or promote a better one.
remote_run is the main factor in source: use a warm lease, improve pnpm store reuse, or shift machine prerequisites into the provider image.
remote_run is the bottleneck in prehydrated: the remote workspace wasn't actually ready, or gateway, browser, or Slack setup is slow.
When artifact copy dominates, check video size and what is inside the artifact directory.

Evidence checklist

A useful PR comment includes:

scenario ID and candidate SHA
GitHub Actions run URL and artifact URL
an inline approval checkpoint screenshot, or a Slack Web screenshot taken from a logged-in warm lease
an inline animated preview when available
links to the full MP4 and a trimmed MP4
pass or fail status along with the report's timing summary

Never commit screenshots or videos to the repository. Keep them inside GitHub Actions artifacts or within the PR comment.

Failure handling

If the workflow fails before the VM runs, start by inspecting the Actions job. Common causes: untrusted candidate_ref, missing environment secrets, or a candidate install or build failure.

If the VM run fails but screenshots were copied back, inspect:

cat mantis-slack-desktop-smoke-report.md
cat mantis-slack-desktop-smoke-summary.json
cat slack-desktop-command.log
cat openclaw-gateway.log
cat chrome.log
cat ffmpeg.log

If the run kept the lease, open VNC using the report's crabbox vnc ... command, then stop the lease when finished:

crabbox stop --provider aws <cbx_id-or-slug>

If Slack login expired, fix it in VNC on a kept lease and rerun with --lease-id. Do not bake that browser profile into a provider image.