WattRat QA Playbook
Defines a maintainer QA workflow with a regression command and manual validation steps for different change types.
What this file does
Defines a maintainer QA workflow with a regression command and manual validation steps for different change types.
When to use it
- You maintain a project with CLI, API, and GUI components
- You need a lightweight QA checklist before risky merges
- You want to separate automated regression from manual system-state tests
- You are onboarding a new maintainer to your QA process
Assumes this stack
WattRat QA Playbook
This complements qa/README.md with a maintainer-focused workflow.
Default Regression Command
./qa/run-qa.sh
Run this after any change that affects behavior.
What run-qa.sh Covers
- CLI sanity through
qa/cli-tests.sh - API sanity through
qa/api-tests.py - automatic temporary GUI startup on port
7172whenWATTRAT_GUI_URLis not provided
What It Does Not Fully Prove
- real benchmark accuracy
- install and revert safety on a real machine
- privileged profile application side effects
- browser UX correctness
- hardware-specific power improvements
That means QA in this repo has two layers:
- regression coverage from
qa/run-qa.sh - targeted manual validation for system-state and hardware-sensitive changes
Recommended QA By Change Type
Docs only
- verify referenced files and commands exist
GUI changes
./qa/run-qa.sh- Playwright MCP smoke test for the touched flow
Variant or validator changes
./run.sh variant list./run.sh variant validate <touched-variant>./qa/run-qa.sh
Benchmark changes
./qa/run-qa.sh- inspect cleanup paths
- run a short manual benchmark smoke test only when safe
Install, revert, or service-control changes
./qa/run-qa.sh- manual root validation on a non-critical machine when possible
- explicitly verify backup and restore assumptions
Manual Smoke Tests Worth Reusing
GUI API
python3 gui/server.py --port=7172
python3 qa/api-tests.py
Variant sanity
./run.sh variant list
./run.sh variant validate wattrat-balanced
./run.sh variant validate stock
Benchmark smoke test
sudo ./run.sh benchmark --profile=current --idle-secs=10 --load-secs=10 --repeats=1
Use only when a short real run is appropriate.
Failure Triage
- If CLI QA fails, inspect
run.sh, variant validation, and schema compatibility first. - If API QA fails, inspect
gui/server.pyand any changed JSON shapes. - If a system-state change looks risky but QA is green, call that out. The QA suite is intentionally light.
Release Mindset
Before a release or a risky merge:
- run
./qa/run-qa.sh - validate the main touched path manually
- confirm docs still match commands and file locations
- note any skipped root or hardware checks explicitly
What's inside
7 sections, 5 change-type recommendations, 3 manual smoke tests, 1 failure triage guide
Change this for your project
- Replace
WattRatwith your project name - Replace
./qa/run-qa.shwith your own regression script path - Replace
gui/server.pywith your GUI server entry point - Replace
./run.shwith your project's CLI entry point
Where it goes
Keep it in your repository where the agent or team that needs it will read it.
Related Documents
Video/Content Producer
Defines a video/content producer role with script structures, hook formulas, storyboard templates, and platform-specific formats for short-form and long-form video production.
Media Handling Playbook - Zyeuté v3
Documents upload, storage, optimization, delivery, and edge-case handling for images, videos, and audio in a social app.
Visual Truth Engine: Product-Market Fit & Go-to-Market Strategy
Outlines a product-market fit strategy and go-to-market plan for a B2B SaaS tool that automates proof asset creation for video creators.
Devin Triage Playbook
Defines a 7-phase automated triage playbook for Devin to analyze alerts, find root causes, implement fixes, and notify teams.