Full Release Validation: Stages, Workflows, and Evidence

Full Release Validation serves as the umbrella for release product validation. Most of the workload lives in child workflows, so a failed box can be retried without kicking off the entire release from scratch. Prior to freezing the Code SHA, execute release preparation; it updates Control UI locale output when the background bot has not yet delivered it, then applies the same strict zero-fallback check that release CI enforces.

Freeze the product-complete pre-changelog commit as the Code SHA, pick a single trusted workflow commit as the Tooling SHA, and then execute:

pnpm ci:full-release \
  --sha <code-sha> \
  --target-ref release/YYYY.M.PATCH

For cross-OS onboarding and the end-to-end agent turn, provider also supports anthropic or minimax. The helper derives the beta profile from alpha/beta package versions, defaulting to stable otherwise. Alternative workflow inputs go through -f key=value; reserve -f release_profile=full for the broad advisory sweep alone. By default, fail_fast is set to false, which lets dispatched child workflows finish and surface independent failures together. If the shorter first-failure cancellation route is preferred, pass -f fail_fast=true.

A temporary release-ci/* ref pinned to the Tooling SHA gets created by the helper, which also passes the Validation SHA as both the candidate ref and expected_sha, removing the temporary ref once validation wraps up. For product validation, the Validation SHA matches the Code SHA; for changelog-only validation, it matches the Release SHA. It does not act as a third release identity. Before child dispatch, the workflow rejects expected SHAs that are malformed or mismatched. Each child must report the same Tooling SHA. To force a fresh run, pass -f reuse_evidence=false; to pick an older compatible workflow commit still reachable from the current origin/main, pass --workflow-sha <trusted-main-sha>. A pinned Tooling SHA that fails to declare the expected_sha dispatch input gets rejected by the helper, which never quietly swaps in newer tooling. The workflow itself never creates or updates repository refs.

Extended-stable exception

Extended-stable publishing demands a run whose workflow and target both sit on the canonical branch:

RELEASE_SHA="$(git rev-parse HEAD)"
gh workflow run full-release-validation.yml \
  --ref extended-stable/YYYY.M.33 \
  -f ref=extended-stable/YYYY.M.33 \
  -f expected_sha="$RELEASE_SHA" \
  -f release_profile=stable

Avoid pnpm ci:full-release and release-ci/*. Publishing ties the run's branch, head/target SHA, manifest workflowRef, ID, and attempt to the canonical branch and release commit.

Backport product failures; for frozen-target tooling, apply the smallest behavior-preserving fix; retry provider, approval, or runner failures without touching the source. Any branch alteration calls for a complete new run. Required package, installer, update, channel, or live behavior cannot be omitted simply because the target is old.

For a regular release with a green Code SHA, generate and commit only CHANGELOG.md. That new commit becomes the Release SHA. Run the same helper against the Release SHA. Product evidence is reused solely when GitHub confirms the Release SHA descends from the Code SHA and the full changed path set equals exactly CHANGELOG.md; npm preflight and package/install acceptance still execute on the Release SHA.

The current inputs map to the conceptual phases:

• beta-publish: release_profile=beta, run_release_soak=false
• postpublish-confidence: the exact published package plus run_release_soak=true or explicit focused groups
• stable-publish: release_profile=stable

Beta-publish all leaves out broad live/E2E soak and QA-live lanes. Stable and full always include the soak. Stable publication refuses a validation manifest that lacks soak and blocking product-performance evidence.

Package Acceptance normally builds the candidate tarball from the resolved ref, covering full-SHA runs dispatched with pnpm ci:full-release. After a beta publish, pass release_package_spec=openclaw@YYYY.M.PATCH-beta.N to reuse the shipped npm package across release checks, Package Acceptance, cross-OS, release-path Docker, and package Telegram. Use package_acceptance_package_spec only when Package Acceptance should deliberately verify a different package. The Codex plugin live package lane follows the same pattern: published release_package_spec values drive codex_plugin_spec=npm:@openclaw/codex@<version>; SHA/artifact runs bundle extensions/codex from the selected ref; and operators may set codex_plugin_spec directly for npm:, npm-pack:, or git: plugin sources. That lane grants the explicit Codex CLI install approval the plugin requires, then executes Codex CLI preflight and same-session OpenAI agent turns. Its final zero-retry, medium-thinking turn sends visible progress with Codex final omitted, reads randomized workspace inputs, writes their exact artifact, and sends explicit completion. This catches the v2026.7.1 regression where an ordinary progress send ended the turn prematurely.

Reserve -f skip_package_telegram_e2e=true for when the release owner explicitly postpones the Package Acceptance Telegram E2E to a follow-up beta. The input gets rejected for stable and full, is recorded in validation evidence, and does not disable the focused rerun_group=npm-telegram workflow.

Top-level stages

For rerun_group=all, a Check for reusable validation evidence job runs first. It searches for the newest prior green full validation matching the same release profile, effective soak setting, and validation inputs. Exact-target reruns rely on exact-target-full-validation-v1. A descendant whose complete delta is exactly CHANGELOG.md uses changelog-only-release-v1; every product lane is skipped, and the verifier independently rechecks the GitHub commit comparison, immutable parent artifact, child runs, and dispatch logs. Any other target change demands a fresh Code SHA validation. Pass reuse_evidence=false to force a fresh full run. Evidence reuse happens only from main or a canonical SHA-pinned release-ci/* ref whose workflow commit stays on trusted main lineage; other workflow refs execute the selected lanes fresh.

Fresh package-facing validation prepares one immutable tarball and one Docker image artifact before dispatching Plugin Prerelease and OpenClaw Release Checks. Both children verify the same package SHA, artifact IDs, service digests, producer run attempt, and Docker archive digest before use. The package-independent bare Docker layer relies on a content-addressed GHCR cache; candidate-specific images stay immutable GitHub artifacts. Focused runs with an explicit published package spec keep the existing package path instead.

For rerun_group=all, a Verify Docker runtime image assets job compiles the runtime-assets Docker target using OPENCLAW_EXTENSIONS=diagnostics-otel,codex. This runs concurrently with the remaining stages and is mandated by the umbrella verifier; lanes no longer pause for it before launching. A more limited rerun_group bypasses this preliminary check.

The umbrella always dispatches product performance in artifact-only mode. OpenClaw Performance permits report publication only for scheduled runs or a manual dispatch that explicitly sets publish_reports=true. The artifact-only guard must complete successfully, proving the publisher job stayed skipped. Fresh and reused evidence records controls.performanceReportPublication=artifact-only; the verifier and reuse selector reject evidence without the matching normalized performance-child proof.

The verifier uploads the canonical manifest as full-release-validation-<run-id>-<run-attempt>. Evidence tooling validates its artifact ID, digest, producer run, and attempt before downloading that exact artifact ID. It caps the downloaded ZIP, verifies its bytes against the REST sha256: digest, and streams the only allowed bounded manifest entry without extracting the archive. A stable-name alias remains temporarily for older publish consumers. The verifier always prefers the attempt-qualified artifact; as a transition, it accepts the stable name only for an attempt-1 manifest v2 producer. It rejects that legacy name for later attempts and manifest v3.

Concurrency is keyed by Validation SHA, Tooling SHA, and rerun group and does not cancel an older run. Parent cancellation or timeout leaves an adopted identity-checked child running. Cancel that exact child explicitly when it is no longer useful.

Release checks stages

OpenClaw Release Checks is the largest child workflow. It resolves the target once and validates the umbrella's shared package artifact when available. A direct or focused dispatch prepares its own release-package-under-test artifact when package or Docker-facing stages need it.

Docker release-path chunks

With live_suite_filter left blank, the Docker release-path stage executes these chunks:

When a single Docker lane fails, apply docker_lanes=<lane[,lane]> on the reusable live/E2E workflow. Per-lane rerun commands, complete with package artifact and image reuse inputs where available, ship with the release artifacts.

Release profiles

Within release checks, release_profile largely dictates live/provider breadth. Normal full CI, Plugin Prerelease, install smoke, package acceptance, and QA parity remain untouched. Exhaustive repo/live E2E, Docker release-path, and QA-live soak coverage always run under stable and full profiles. The beta profile only adds those lanes when run_release_soak=true is set or an explicit qa or qa-live rerun happens. Since Package Acceptance supplies the canonical package Telegram E2E for every candidate, the umbrella avoids duplicating that live poller.

Full-only additions

stable causes these suites to be skipped, while full brings them in:

stable bundles native-live-src-gateway-profiles-anthropic-smoke and native-live-src-gateway-profiles-opencode-go-smoke; the wider Anthropic and OpenCode Go model shards are what full relies on instead. Focused reruns retain access to the aggregate native-live-src-gateway-profiles-anthropic or native-live-src-gateway-profiles-opencode-go handles.

Focused reruns

To sidestep rerunning unrelated release boxes, use rerun_group:

When a single live suite fails, pair live_suite_filter with rerun_group=live-e2e. The reusable live/E2E workflow defines the valid filter ids, which cover docker-live-models, live-gateway-docker, live-gateway-anthropic-docker, live-gateway-google-docker, live-gateway-minimax-docker, live-gateway-advisory-docker, live-cli-backend-docker, live-acp-bind-docker, and live-codex-harness-docker.

To rerun a specific QA transport, set rerun_group=qa-live and pick the canonical selector: qa-live-matrix, qa-live-buzz, qa-live-telegram, qa-live-discord, qa-live-whatsapp, or qa-live-slack.

The live-gateway-advisory-docker handle reruns all three provider shards as a group, so it still dispatches to every advisory Docker gateway job.

If one cross-OS lane fails, use cross_os_suite_filter with rerun_group=cross-os. The filter takes an OS id, a suite id, or an OS/suite combination, such as windows/packaged-upgrade, windows, or packaged-fresh. Cross-OS summaries show per-phase timings for packaged upgrade lanes, and long-running commands emit heartbeat lines so a hung update is apparent before the job times out.

QA release-check failures hold up normal release validation only for the chosen Matrix, Telegram, and QA runtime tool coverage lanes. QA parity, runtime parity, and the gated Discord, WhatsApp, and Slack live lanes are advisory, publishing status artifacts without halting the release verifier. Tideclaw alpha runs may still treat non-package-safety release-check lanes as advisory. With release_profile=beta, the Run repo/live E2E validation live-provider suites are advisory: third-party model deployments shift underneath a release, so beta surfaces their failures as warnings while stable and full profiles keep them blocking. When live_suite_filter explicitly asks for a gated QA live lane such as Discord, WhatsApp, or Slack, the matching OPENCLAW_RELEASE_QA_*_LIVE_CI_ENABLED repo variable must be enabled; otherwise input capture fails instead of silently skipping the lane. Rerun rerun_group=qa, qa-parity, or qa-live when you need fresh QA evidence.

Evidence to keep

Use the Full Release Validation summary as the release-level index. It links child run ids and includes slowest-job tables. Classify failures as product, harness/tooling/provenance, infrastructure/credential, or wrapper. Only a confirmed product failure changes the Code SHA. Use one diagnosis, one fix when needed, and one narrow retry, then reassess; do not automatically rerun all. Narrow evidence is not publish authorization by itself.

For a regular release, record both Code SHA and Release SHA, the reuse policy and changed-path set, the green Code SHA parent run, and the lightweight Release SHA parent run. For extended-stable, record the canonical branch, exact release SHA, fresh parent run id and attempt, workflow ref, every child run, and any frozen-target compatibility repair or intentional omission.

Useful artifacts:

• release-package-under-test from OpenClaw Release Checks
• Docker release-path artifacts under .artifacts/docker-tests/
• Package Acceptance package-under-test and Docker acceptance artifacts
• Cross-OS release-check artifacts for each OS and suite
• QA parity, runtime parity, and selected Matrix, Buzz, Telegram, Discord, WhatsApp, or Slack artifacts

Workflow files

• .github/workflows/full-release-validation.yml
• .github/workflows/openclaw-release-checks.yml
• .github/workflows/openclaw-live-and-e2e-checks-reusable.yml
• .github/workflows/plugin-prerelease.yml
• .github/workflows/install-smoke.yml
• .github/workflows/install-smoke-reusable.yml
• .github/workflows/openclaw-cross-os-release-checks-reusable.yml
• .github/workflows/package-acceptance.yml
• .github/workflows/openclaw-performance.yml
• .github/workflows/npm-telegram-beta-e2e.yml