WattRat QA Playbook

This complements qa/README.md with a maintainer-focused workflow.

Default Regression Command

./qa/run-qa.sh

Run this after any change that affects behavior.

What run-qa.sh Covers

• CLI sanity through qa/cli-tests.sh
• API sanity through qa/api-tests.py
• automatic temporary GUI startup on port 7172 when WATTRAT_GUI_URL is not provided

What It Does Not Fully Prove

• real benchmark accuracy
• install and revert safety on a real machine
• privileged profile application side effects
• browser UX correctness
• hardware-specific power improvements

That means QA in this repo has two layers:

• regression coverage from qa/run-qa.sh
• targeted manual validation for system-state and hardware-sensitive changes

Recommended QA By Change Type

Docs only

• verify referenced files and commands exist

GUI changes

• ./qa/run-qa.sh
• Playwright MCP smoke test for the touched flow

Variant or validator changes

• ./run.sh variant list
• ./run.sh variant validate <touched-variant>
• ./qa/run-qa.sh

Benchmark changes

• ./qa/run-qa.sh
• inspect cleanup paths
• run a short manual benchmark smoke test only when safe

Install, revert, or service-control changes

• ./qa/run-qa.sh
• manual root validation on a non-critical machine when possible
• explicitly verify backup and restore assumptions

Manual Smoke Tests Worth Reusing

GUI API

python3 gui/server.py --port=7172
python3 qa/api-tests.py

Variant sanity

./run.sh variant list
./run.sh variant validate wattrat-balanced
./run.sh variant validate stock

Benchmark smoke test

sudo ./run.sh benchmark --profile=current --idle-secs=10 --load-secs=10 --repeats=1

Use only when a short real run is appropriate.

Failure Triage

• If CLI QA fails, inspect run.sh, variant validation, and schema compatibility first.
• If API QA fails, inspect gui/server.py and any changed JSON shapes.
• If a system-state change looks risky but QA is green, call that out. The QA suite is intentionally light.

Release Mindset

Before a release or a risky merge:

1. run ./qa/run-qa.sh
2. validate the main touched path manually
3. confirm docs still match commands and file locations
4. note any skipped root or hardware checks explicitly