OSS Forensics: GitHub Supply-Chain Attack Investigation Guide

This skill runs a seven-phase, multi-agent investigation into open-source supply chain attacks. You would reach for it when a dependency goes missing, a maintainer account behaves oddly, or a force-push hides a leaked secret. It turns GitHub Archive, the Wayback Machine, the GitHub API, and a local clone into a single evidence-backed narrative, complete with a forensic report you can hand to a maintainer or registry.

What it does

The skill is a structured framework, not a single script. It splits an investigation into seven phases, from parsing the initial request down to a final report. Along the way it spawns up to five specialist sub-agents, each locked to one data source. One reads the local git history, another talks only to the GitHub REST API, a third queries the Wayback Machine CDX API, a fourth runs BigQuery against GitHub Archive, and the last enriches the indicators you started with. Each agent writes evidence into a shared JSON store, and a validator checks that every claim in the final report points at a real evidence ID.

The design assumes you are investigating a public repository. The GitHub Archive and Wayback Machine only cover public activity, and the GitHub API works best with a token. The skill is defensive: it exists to protect open-source software, not to attack it. The ethical guidelines in the source are explicit about that, and the anti-hallucination rules are the backbone of the whole process.

Before you start

This is an optional skill, installed on demand. It lives at optional-skills/security/oss-forensics and is version 1.0.0, authored by Teknium (teknium1) for Hermes Agent. It is MIT licensed and runs on Linux, macOS, and Windows.

You need a working Python environment for the evidence store script. The skill calls python SKILL_DIR/scripts/evidence-store.py, where SKILL_DIR is the root of the skill installation, for example ~/.hermes/skills/security/oss-forensics/ or the optional-skills/ equivalent. All script and template references are relative to that directory.

For the GitHub API investigator, authenticate. Unauthenticated requests are capped at 60 per hour, which is unusable for real work. Authenticated requests allow 5,000 per hour, and you can authenticate with a GITHUB_TOKEN environment variable or the gh CLI. The GH Archive investigator needs Google Cloud credentials with BigQuery access, set up via gcloud auth application-default login. If those credentials are missing, skip that investigator and note it in the report.

Phase 0: Initialization

Every investigation starts the same way. Create a working directory named after the repository, initialize the evidence store, and copy in the report template.

mkdir investigation_$(echo "REPO_NAME" | tr '/' '_')
cd investigation_$(echo "REPO_NAME" | tr '/' '_')

python SKILL_DIR/scripts/evidence-store.py --store evidence.json list

cp SKILL_DIR/templates/forensic-report.md ./investigation-report.md

Then create an iocs.md file to track Indicators of Compromise as they are discovered, and record the investigation start time, the target repository, and the stated investigation goal. This bookkeeping matters later, when you need to show chain of custody.

Phase 1: Prompt Parsing and IOC Extraction

The first real step is to turn the user's request into a structured list of targets. Parse the prompt for the target repository (owner/repo), target actors (GitHub handles, email addresses), the time window of interest (commit date ranges, PR timestamps), and any provided IOCs: commit SHAs, file paths, package names, IP addresses, domains, API keys or tokens, and malicious URLs. Also note any linked vendor security reports or blog posts.

You can do this with reasoning alone, or use execute_code for regex extraction from large text blocks. The output populates iocs.md. Each IOC must have a type from this list: COMMIT_SHA, FILE_PATH, API_KEY, SECRET, IP_ADDRESS, DOMAIN, PACKAGE_NAME, ACTOR_USERNAME, MALICIOUS_URL, OTHER. Each also needs a value and a source, either user-provided or inferred. The full taxonomy lives in evidence-types.md.

Phase 2: Parallel Evidence Collection

This is the heart of the skill. Spawn up to five specialist investigator sub-agents using delegate_task in batch mode, with a maximum of three concurrent. Each investigator has a single data source and must not mix sources. Pass the IOC list from Phase 1 and the investigation time window in the context field of each delegated task.

Investigator 1: Local Git Investigator

This agent works only on the local git repository. It clones the repo, dumps the full commit log, looks for dangling commits, checks the reflog, lists branches, finds suspicious binary additions, and checks GPG signatures.

# Clone repository
git clone https://github.com/OWNER/REPO.git target_repo && cd target_repo

# Full commit log with stats
git log --all --full-history --stat --format="%H|%ae|%an|%ai|%s" > ../git_log.txt

# Detect force-push evidence (orphaned/dangling commits)
git fsck --lost-found --unreachable 2>&1 | grep commit > ../dangling_commits.txt

# Check reflog for rewritten history
git reflog --all > ../reflog.txt

# List ALL branches including deleted remote refs
git branch -a -v > ../branches.txt

# Find suspicious large binary additions
git log --all --diff-filter=A --name-only --format="%H %ai" -- "*.so" "*.dll" "*.exe" "*.bin" > ../binary_additions.txt

# Check for GPG signature anomalies
git log --show-signature --format="%H %ai %aN" > ../signature_check.txt 2>&1

Collect evidence with python SKILL_DIR/scripts/evidence-store.py add. Each dangling commit SHA becomes evidence of type git. Force-push evidence from the reflog, unsigned commits from verified contributors, and suspicious binary additions all go in as type git. The recovery-techniques.md reference explains how to access force-pushed commits.

Investigator 2: GitHub API Investigator

This agent talks only to the GitHub REST API. It pulls commits, pull requests, issues, contributors, events, releases, and details on specific commits. The curl commands below write JSON files you can inspect later.

# Commits (paginated)
curl -s "https://api.github.com/repos/OWNER/REPO/commits?per_page=100" > api_commits.json

# Pull Requests including closed/deleted
curl -s "https://api.github.com/repos/OWNER/REPO/pulls?state=all&per_page=100" > api_prs.json

# Issues
curl -s "https://api.github.com/repos/OWNER/REPO/issues?state=all&per_page=100" > api_issues.json

# Contributors and collaborator changes
curl -s "https://api.github.com/repos/OWNER/REPO/contributors" > api_contributors.json

# Repository events (last 300)
curl -s "https://api.github.com/repos/OWNER/REPO/events?per_page=100" > api_events.json

# Check specific suspicious commit SHA details
curl -s "https://api.github.com/repos/OWNER/REPO/git/commits/SHA" > commit_detail.json

# Releases
curl -s "https://api.github.com/repos/OWNER/REPO/releases?per_page=100" > api_releases.json

# Check if a specific commit exists (force-pushed commits may 404 on commits/ but succeed on git/commits/)
curl -s "https://api.github.com/repos/OWNER/REPO/commits/SHA" | jq .sha

The real value is in cross-referencing. A PR that exists in the archive but is missing from the API is evidence of deletion. A contributor who appears in archive events but not in the contributors list suggests permission revocation. A commit in archive PushEvents but not in the API commit list points to force-push or deletion. Flag these discrepancies as evidence.

Investigator 3: Wayback Machine Investigator

The Wayback Machine can recover deleted GitHub pages: READMEs, issues, PRs, releases, and wiki pages. This agent queries the CDX API only.

# Search for archived snapshots of the repo main page
curl -s "https://web.archive.org/cdx/search/cdx?url=github.com/OWNER/REPO&output=json&limit=100&from=YYYYMMDD&to=YYYYMMDD" > wayback_main.json

# Search for a specific deleted issue
curl -s "https://web.archive.org/cdx/search/cdx?url=github.com/OWNER/REPO/issues/NUM&output=json&limit=50" > wayback_issue_NUM.json

# Search for a specific deleted PR
curl -s "https://web.archive.org/cdx/search/cdx?url=github.com/OWNER/REPO/pull/NUM&output=json&limit=50" > wayback_pr_NUM.json

# Fetch the best snapshot of a page
# Use the Wayback Machine URL: https://web.archive.org/web/TIMESTAMP/ORIGINAL_URL
# Example: https://web.archive.org/web/20240101000000*/github.com/OWNER/REPO

# Advanced: Search for deleted releases/tags
curl -s "https://web.archive.org/cdx/search/cdx?url=github.com/OWNER/REPO/releases/tag/*&output=json" > wayback_tags.json

# Advanced: Search for historical wiki changes
curl -s "https://web.archive.org/cdx/search/cdx?url=github.com/OWNER/REPO/wiki/*&output=json" > wayback_wiki.json

Collect archived snapshots of deleted issues or PRs with their content, historical README versions, and any content that exists in the archive but is missing from the current GitHub state. The github-archive-guide.md reference covers CDX API parameters.

Investigator 4: GH Archive / BigQuery Investigator

GitHub Archive is a tamper-proof record of all public GitHub events, queryable through BigQuery. This is the most powerful source for detecting force-pushes and deletions, because the archive never rewrites history.

Before running any query, you must estimate its cost with --dry_run. The rules are mandatory: always dry-run, use _TABLE_SUFFIX to filter by date range, select only the columns you need, and add a LIMIT unless you are aggregating.

# Template: safe BigQuery query for PushEvents to OWNER/REPO
bq query --use_legacy_sql=false --dry_run "
SELECT created_at, actor.login, payload.commits, payload.before, payload.head,
       payload.size, payload.distinct_size
FROM \`githubarchive.month.*\`
WHERE _TABLE_SUFFIX BETWEEN 'YYYYMM' AND 'YYYYMM'
  AND type = 'PushEvent'
  AND repo.name = 'OWNER/REPO'
LIMIT 1000
"
# If cost is acceptable, re-run without --dry_run

# Detect force-pushes: zero-distinct_size PushEvents mean commits were force-erased
# payload.distinct_size = 0 AND payload.size > 0 → force push indicator

# Check for deleted branch events
bq query --use_legacy_sql=false "
SELECT created_at, actor.login, payload.ref, payload.ref_type
FROM \`githubarchive.month.*\`
WHERE _TABLE_SUFFIX BETWEEN 'YYYYMM' AND 'YYYYMM'
  AND type = 'DeleteEvent'
  AND repo.name = 'OWNER/REPO'
LIMIT 200
"

A PushEvent with payload.size > 0 and payload.distinct_size = 0 is a strong force-push indicator. DeleteEvents for branches or tags, WorkflowRunEvents for suspicious CI/CD automation, and PushEvents that precede a gap in the git log are all evidence. The github-archive-guide.md reference lists all 12 event types and query patterns.

Investigator 5: IOC Enrichment Investigator

This agent enriches the IOCs from Phase 1 using passive public sources only. It never executes code from the target repository.

For each commit SHA, try recovery via the direct GitHub URL github.com/OWNER/REPO/commit/SHA.patch. For each domain or IP, check passive DNS and WHOIS records via web_extract on public WHOIS services. For each package name, check npm or PyPI for matching malicious package reports. For each actor username, check the GitHub profile, contribution history, and account age. Recover force-pushed commits using the three methods in recovery-techniques.md.

Phase 3: Evidence Consolidation

Once all investigators finish, bring everything together. Run the evidence store list command to see what was collected.

python SKILL_DIR/scripts/evidence-store.py --store evidence.json list

For each piece of evidence, verify the content_sha256 hash matches the original source. Then group evidence three ways: by timeline (sort all timestamped evidence chronologically), by actor (GitHub handle or email), and by IOC (link evidence to the IOC it relates to). Identify discrepancies: items present in one source but absent in another are key deletion indicators. Finally, flag each piece as [VERIFIED] if confirmed from two or more independent sources, or [UNVERIFIED] if only a single source supports it.

Phase 4: Hypothesis Formation

A hypothesis must state a specific claim, cite at least two evidence IDs that support it, identify what evidence would disprove it, and be labeled [HYPOTHESIS] until validated. The source gives common templates: maintainer compromise, dependency confusion, CI/CD injection, typosquatting, and credential leak. For each hypothesis, spawn a delegate_task sub-agent to look for disconfirming evidence before you confirm anything.

Phase 5: Hypothesis Validation

The validator sub-agent mechanically checks every hypothesis. It extracts all cited evidence IDs, verifies each exists in evidence.json (a missing ID is a hard failure and the hypothesis is rejected as potentially fabricated), verifies each [VERIFIED] piece came from two or more sources, checks that the timeline depicted by the evidence supports the hypothesis, and looks for alternative explanations: could the same evidence pattern arise from a benign cause?

The output is one of three statuses. VALIDATED means all evidence is cited, verified, logically consistent, and no plausible alternative exists. INCONCLUSIVE means the evidence supports the hypothesis but alternatives exist or evidence is insufficient. REJECTED means missing evidence IDs, unverified evidence cited as fact, or logical inconsistency. Rejected hypotheses feed back into Phase 4 for refinement, up to three iterations.

Phase 6: Final Report Generation

Populate investigation-report.md using the template in forensic-report.md. The mandatory sections are an executive summary with a one-paragraph verdict (Compromised, Clean, or Inconclusive) and a confidence level; a chronological timeline of significant events with evidence citations; validated hypotheses with status and supporting evidence IDs; an evidence registry table of all EV-XXXX entries with source, type, and verification status; the IOC list; a chain of custody showing how evidence was collected, from what sources, and at what timestamps; and recommendations for immediate mitigations or monitoring.

Two rules are absolute. Every factual claim must have at least one [EV-XXXX] citation, and all secrets or credentials must be redacted to [REDACTED].

Phase 7: Completion

Run the final evidence count, archive the full investigation directory, and present the report. If compromise is confirmed, list immediate mitigations such as rotating credentials, pinning dependency hashes, and notifying affected users. Identify affected versions and packages. If a public package is involved, coordinate with the package registry and note any disclosure obligations.

Anti-Hallucination Guardrails

These rules are not optional. The source says violating them invalidates the report, so read them before every investigation step.

1. Evidence-First Rule: Every claim in any report, hypothesis, or summary MUST cite at least one evidence ID (EV-XXXX). Assertions without citations are forbidden.
2. STAY IN YOUR LANE: Each sub-agent (investigator) has a single data source. Do NOT mix sources. The GH Archive investigator does not query the GitHub API, and vice versa. Role boundaries are hard.
3. Fact vs. Hypothesis Separation: Mark all unverified inferences with [HYPOTHESIS]. Only statements verified against original sources may be stated as facts.
4. No Evidence Fabrication: The hypothesis validator MUST mechanically check that every cited evidence ID actually exists in the evidence store before accepting a hypothesis.
5. Proof-Required Disproval: A hypothesis cannot be dismissed without a specific, evidence-backed counter-argument. "No evidence found" is not sufficient to disprove, it only makes a hypothesis inconclusive.
6. SHA/URL Double-Verification: Any commit SHA, URL, or external identifier cited as evidence must be independently confirmed from at least two sources before being marked as verified.
7. Suspicious Code Rule: Never run code found inside the investigated repository locally. Analyze statically only, or use execute_code in a sandboxed environment.
8. Secret Redaction: Any API keys, tokens, or credentials discovered during investigation must be redacted in the final report. Log them internally only.

Example Scenarios

The source gives three concrete scenarios. In Scenario A, a malicious package internal-lib-v2 is uploaded to NPM with a higher version than the internal one. The investigator tracks when this package was first seen and whether any PushEvents in the target repo updated package.json to this version. In Scenario B, a long-term contributor's account is used to push a backdoored .github/workflows/build.yml. The investigator looks for PushEvents from this user after a long period of inactivity or from a new IP or location, if detectable via BigQuery. In Scenario C, a developer accidentally commits a production secret, then force-pushes to "fix" it. The investigator uses git fsck and GH Archive to recover the original commit SHA and verify what was leaked.

Ethical Use Guidelines

This skill is for defensive security investigation only. The source forbids using it for harassment or stalking of contributors or maintainers, doxing (correlating GitHub activity to real identities for malicious purposes), competitive intelligence against proprietary or internal repositories without authorization, and false accusations based on unvalidated evidence. Investigations should follow the principle of minimal intrusion: collect only the evidence necessary to validate or refute the hypothesis. When publishing results, follow responsible disclosure and coordinate with affected maintainers first.

If the investigation reveals a genuine compromise, follow the coordinated vulnerability disclosure process: notify the repository maintainers privately first, allow reasonable time for remediation (typically 90 days), coordinate with package registries like npm or PyPI if published packages are affected, and file a CVE if appropriate.

API Rate Limiting

Rate limits will interrupt large investigations if you do not manage them. The GitHub REST API allows 5,000 authenticated requests per hour and only 60 unauthenticated, which is unusable. Always authenticate with export GITHUB_TOKEN=ghp_... or use the gh CLI, which auto-authenticates. Use conditional requests with If-None-Match or If-Modified-Since headers to avoid consuming quota on unchanged data. For paginated endpoints, fetch all pages in sequence, do not parallelize against the same endpoint. Check the X-RateLimit-Remaining header, and if it is below 100, pause until the X-RateLimit-Reset timestamp. BigQuery has its own quotas, 10 TiB per day free tier, so always dry-run first. The Wayback Machine CDX API has no formal rate limit, but be courteous and keep to 1-2 requests per second. If you get rate-limited mid-investigation, record the partial results in the evidence store and note the limitation in the report.

When not to use it

The source does not list explicit exclusions, but the ethical guidelines imply boundaries. Do not use this skill for private or internal repositories without authorization, and do not use it to build profiles of individuals. It is also not the right tool if you lack the prerequisites: without BigQuery credentials you lose the most powerful source, and without a GitHub token the API investigator is effectively useless. For a quick look at a single suspicious commit, a plain git log and a GitHub search are faster.

Limits and gotchas

The biggest gotcha is the anti-hallucination rules. They are strict, and the source says violating them invalidates the report. The evidence store is the single source of truth; if an evidence ID is missing, the validator rejects the hypothesis. The role boundaries are hard, so do not let one investigator call the GitHub API when it should only query BigQuery. The cost optimization rules for BigQuery are mandatory, not suggestions. And never run code from the investigated repository locally, no matter how tempting it is to test a suspicious script.

What pairs with this

The source links several reference materials that extend this skill. github-archive-guide.md covers BigQuery queries, the CDX API, and all 12 event types. evidence-types.md defines the IOC taxonomy and evidence source types. recovery-techniques.md explains how to recover deleted commits, PRs, and issues. investigation-templates.md provides pre-built hypothesis templates per attack type. The evidence-store.py script manages the evidence JSON store, and forensic-report.md is the structured report template. Together they form a complete toolkit for supply-chain forensics.