Karpathy's LLM Wiki Skill: Build a Persistent Markdown Knowledge Base
Karpathy's LLM Wiki: build/query interlinked markdown KB.
Written by Neura Market from the official Hermes Agent documentation for Llm Wiki. Commands, paths, and version numbers are reproduced from the source unchanged.
Read the official documentationKarpathy's LLM Wiki is a pattern for building a persistent, compounding knowledge base out of interlinked markdown files. It follows the approach Andrej Karpathy described in his public gist, and it is a good fit when you want to accumulate understanding over time rather than answer one-off questions.
Traditional RAG retrieves relevant chunks for each query and reassembles an answer on the spot. That works for many use cases, but it means the same knowledge gets rediscovered over and over. The wiki takes the opposite route: it compiles knowledge once, then keeps that compilation current. Cross-references between topics are already in place, contradictions have already been flagged, and the synthesis reflects everything that has been ingested so far. The result is a knowledge base that gets more valuable as it grows, because each new piece of information builds on what is already there.
The pattern splits the work between a human and an agent. The human curates the sources and decides what to analyze. The agent handles the mechanical parts: summarizing, cross-referencing, filing, and keeping the whole thing consistent. In practice, this means you spend your time choosing what matters and asking the right questions, while the agent keeps the notes organized and up to date.
Reach for this when you are tracking a topic that evolves, when you need to connect ideas across many documents, or when you want a durable reference that does not require re-querying every time. It pairs well with a workflow where you regularly add new material and expect the knowledge base to reflect the latest state. The main caveat is that it requires ongoing curation: the human side of the division of labor is not optional. If you do not feed it good sources and clear direction, the wiki will not stay useful on its own.
When This Skill Activates
Reach for this skill when the user’s request clearly points at a wiki or knowledge base. The trigger conditions are straightforward, but each one maps to a slightly different workflow, so it helps to recognize them quickly.
- The user asks to create, build, or start a wiki or knowledge base. This is the initial setup path: you are scaffolding something new, not working with existing content.
- The user asks to ingest, add, or process a source into their wiki. This is the content pipeline: bringing in new material, whether a single file, a folder, or an external source, and making it part of the wiki’s structure.
- The user asks a question and an existing wiki is present at the configured path. This is the retrieval path: you are answering from the wiki’s content, not from general knowledge. The key condition is that the wiki actually exists at the expected location, so check that path before assuming you can use it.
- The user asks to lint, audit, or health-check their wiki. This is the maintenance path: verifying the wiki’s integrity, finding broken links, missing metadata, or other issues that accumulate over time.
- The user references their wiki, knowledge base, or “notes” in a research context. This is a broader trigger: even if the request is not explicitly about the wiki, if the user mentions it as a source for research, the skill should activate.
In practice, these triggers often overlap. A user might ask to “add these notes and then check the wiki,” which combines the ingest and audit paths. Or they might ask a question that implies the wiki is the source, even if they do not say “wiki” explicitly. When in doubt, look for the configured path: if a wiki exists there, the skill is likely relevant.
A common mistake is activating the skill for a general knowledge question when no wiki is configured. The third trigger explicitly requires an existing wiki at the configured path, so if that path is missing or empty, the skill should not fire. Similarly, do not treat every mention of “notes” as a research context; the user has to be referencing their own notes as a knowledge source, not just using the word casually.
This skill pairs naturally with other documentation and research workflows. For example, after creating a wiki, you might immediately ingest a set of source files, then run a health check to confirm everything is in order. Recognizing which trigger fired helps you choose the right sequence of actions, but the skill itself is designed to handle all these entry points.
Wiki Location
The wiki's location is controlled by the WIKI_PATH environment variable. You typically set this in your environment file, for example in ${HERMES_HOME:-~/.hermes}/.env. If you don't set it, the wiki defaults to ~/wiki.
The shell snippet below shows how the default is applied: it uses the value of WIKI_PATH if set, otherwise it falls back to $HOME/wiki.
WIKI="${WIKI_PATH:-$HOME/wiki}"
In practice, this means you can point the wiki anywhere you like, such as a directory synced with Dropbox or a folder inside your existing notes. The wiki itself is just a directory of markdown files, so you can open it with Obsidian, VS Code, or any editor you prefer. There's no database or special tooling to install, which keeps things simple and portable. If you're already using a markdown-based note system, you can often point WIKI_PATH at that existing folder and start working immediately.
Architecture: Three Layers
The wiki is organized into three layers, each with a distinct role. The directory layout below shows how these layers map to folders on disk.
wiki/
├── SCHEMA.md # Conventions, structure rules, domain config
├── index.md # Sectioned content catalog with one-line summaries
├── log.md # Chronological action log (append-only, rotated yearly)
├── raw/ # Layer 1: Immutable source material
│ ├── articles/ # Web articles, clippings
│ ├── papers/ # PDFs, arxiv papers
│ ├── transcripts/ # Meeting notes, interviews
│ └── assets/ # Images, diagrams referenced by sources
├── entities/ # Layer 2: Entity pages (people, orgs, products, models)
├── concepts/ # Layer 2: Concept/topic pages
├── comparisons/ # Layer 2: Side-by-side analyses
└── queries/ # Layer 2: Filed query results worth keeping
Layer 1, Raw Sources: This is your immutable source material. The agent reads from these files but never writes to them. Think of this as your evidence base: articles, papers, transcripts, and assets that you or the agent have collected. Because these are never modified, they serve as a reliable reference point. You reach for this layer when you need to preserve original context, such as a PDF you want to cite later or a meeting transcript you don't want accidentally edited.
Layer 2, The Wiki: This is the agent-owned layer. These markdown files are created, updated, and cross-referenced by the agent. They live in subfolders for entities (people, organizations, products, models), concepts, comparisons, and queries. This is where the agent synthesizes information from Layer 1 into structured, navigable knowledge. You use this layer when you want the agent to maintain living documents that evolve as new information arrives.
Layer 3, The Schema: The SCHEMA.md file defines the rules of the game: structure conventions, tagging taxonomy, and domain configuration. It governs how Layer 2 files are organized and how they relate to each other. You adjust this layer when you want to change how the agent categorizes or formats content. In practice, you'll spend most of your time in Layer 2, but you'll occasionally revisit Layer 3 to refine the conventions as your wiki grows.
These layers work together: Layer 1 provides the raw facts, Layer 2 turns them into usable knowledge, and Layer 3 keeps everything consistent. If you're setting up a new wiki, start by defining a minimal SCHEMA.md, then let the agent populate Layer 2 from your raw sources. The separation means you can always trace a claim back to its original source without worrying that the source has been altered.
Resuming an Existing Wiki (CRITICAL, do this every session)
When you return to a wiki that already exists, orient yourself before doing anything else. This is not optional; it is the first step of every session. Skipping it leads to avoidable mistakes.
Start with three reads, in this order:
- Read
SCHEMA.mdto understand the domain, the conventions, and the tag taxonomy. This tells you how the wiki is organized and what rules you must follow. - Read
index.mdto see what pages already exist and their summaries. This prevents you from creating duplicates or missing existing content. - Scan the recent
log.mdentries, the last 20 to 30 lines, to see what has been done recently. This helps you avoid repeating work and stay consistent with recent activity.
The following command performs all three reads at the start of a session. It uses the WIKI environment variable if set, otherwise it defaults to $HOME/wiki:
WIKI="${WIKI_PATH:-$HOME/wiki}"
# Orientation reads at session start
read_file "$WIKI/SCHEMA.md"
read_file "$WIKI/index.md"
read_file "$WIKI/log.md" offset=<last 30 lines>
Only after this orientation should you ingest, query, or lint. Doing otherwise risks:
- Creating duplicate pages for entities that already exist
- Missing cross-references to existing content
- Contradicting the schema's conventions
- Repeating work already logged
For large wikis (100+ pages), also run a quick search_files for the topic at hand before creating anything new. This extra step helps you find relevant existing content quickly, even if it is not obvious from the index.
Initializing a New Wiki
When a user asks to create or start a wiki, follow this sequence. You will typically do this once per project, but you might repeat it if the user wants a separate wiki for a different domain.
- Determine the wiki path. Check the
$WIKI_PATHenvironment variable first. If it is not set, ask the user. If they do not specify, use~/wiki. - Create the directory structure described above. This includes the main wiki folder and the subfolders for raw sources, archive, and meta.
- Ask the user what domain the wiki covers. Be specific. A vague answer like "research" is not enough; you need enough detail to write a useful schema and tag taxonomy.
- Write
SCHEMA.mdcustomized to that domain. Use the template below as a starting point and adapt every bracketed section. - Write the initial
index.mdwith a sectioned header. - Write the initial
log.mdwith a creation entry. - Confirm the wiki is ready and suggest the first sources to ingest. This is a good moment to ask the user for URLs or files they want to start with.
SCHEMA.md Template
Adapt this to the user's domain. The schema constrains agent behavior and ensures consistency across all pages. Without it, different pages might use different naming conventions, frontmatter fields, or linking styles, which makes the wiki hard to navigate and maintain.
# Wiki Schema
## Domain
[What this wiki covers — e.g., "AI/ML research", "personal health", "startup intelligence"]
## Conventions
- File names: lowercase, hyphens, no spaces (e.g., `transformer-architecture.md`)
- Every wiki page starts with YAML frontmatter (see below)
- Use `[[wikilinks]]` to link between pages (minimum 2 outbound links per page)
- When updating a page, always bump the `updated` date
- Every new page must be added to `index.md` under the correct section
- Every action must be appended to `log.md`
- **Provenance markers:** On pages that synthesize 3+ sources, append `^[raw/articles/source-file.md]`
at the end of paragraphs whose claims come from a specific source. This lets a reader trace each
claim back without re-reading the whole raw file. Optional on single-source pages where the
`sources:` frontmatter is enough.
## Frontmatter
```yaml
---
title: Page Title
created: YYYY-MM-DD
updated: YYYY-MM-DD
type: entity | concept | comparison | query | summary
tags: [from taxonomy below]
sources: [raw/articles/source-name.md]
# Optional quality signals:
confidence: high | medium | low # how well-supported the claims are
contested: true # set when the page has unresolved contradictions
contradictions: [other-page-slug] # pages this one conflicts with
---
confidence and contested are optional but recommended for opinion-heavy or fast-moving topics. Lint surfaces contested: true and confidence: low pages for review so weak claims don't silently harden into accepted wiki fact.
raw/ Frontmatter
Raw sources ALSO get a small frontmatter block so re-ingests can detect drift:
---
source_url: https://example.com/article # original URL, if applicable
ingested: YYYY-MM-DD
sha256: <hex digest of the raw content below the frontmatter>
---
The sha256: lets a future re-ingest of the same URL skip processing when content is unchanged, and flag drift when it has changed. Compute over the body only (everything after the closing ---), not the frontmatter itself.
Tag Taxonomy
[Define 10-20 top-level tags for the domain. Add new tags here BEFORE using them.]
Example for AI/ML:
- Models: model, architecture, benchmark, training
- People/Orgs: person, company, lab, open-source
- Techniques: optimization, fine-tuning, inference, alignment, data
- Meta: comparison, timeline, controversy, prediction
Rule: every tag on a page must appear in this taxonomy. If a new tag is needed, add it here first, then use it. This prevents tag sprawl.
Page Thresholds
- Create a page when an entity/concept appears in 2+ sources OR is central to one source
- Add to existing page when a source mentions something already covered
- DON'T create a page for passing mentions, minor details, or things outside the domain
- Split a page when it exceeds ~200 lines, break into sub-topics with cross-links
- Archive a page when its content is fully superseded, move to
_archive/, remove from index
Entity Pages
One page per notable entity. Include:
- Overview / what it is
- Key facts and dates
- Relationships to other entities ([[wikilinks]])
- Source references
Concept Pages
One page per concept or topic. Include:
- Definition / explanation
- Current state of knowledge
- Open questions or debates
- Related concepts ([[wikilinks]])
Comparison Pages
Side-by-side analyses. Include:
- What is being compared and why
- Dimensions of comparison (table format preferred)
- Verdict or synthesis
- Sources
Update Policy
When new information conflicts with existing content:
- Check the dates, newer sources generally supersede older ones
- If genuinely contradictory, note both positions with dates and sources
- Mark the contradiction in frontmatter:
contradictions: [page-name] - Flag for user review in the lint report
### index.md Template
The index is sectioned by type. Each entry is one line: wikilink + summary.
```markdown
# Wiki Index
> Content catalog. Every wiki page listed under its type with a one-line summary.
> Read this first to find relevant pages for any query.
> Last updated: YYYY-MM-DD | Total pages: N
## Entities
<!-- Alphabetical within section -->
## Concepts
## Comparisons
## Queries
Scaling rule: When any section exceeds 50 entries, split it into sub-sections by first letter or sub-domain. When the index exceeds 200 entries total, create a _meta/topic-map.md that groups pages by theme for faster navigation.
log.md Template
# Wiki Log
> Chronological record of all wiki actions. Append-only.
> Format: `## [YYYY-MM-DD] action | subject`
> Actions: ingest, update, query, lint, create, archive, delete
> When this file exceeds 500 entries, rotate: rename to log-YYYY.md, start fresh.
## [YYYY-MM-DD] create | Wiki initialized
- Domain: [domain]
- Structure created with SCHEMA.md, index.md, log.md
Core Operations
1. Ingest
When a user provides a source (URL, file, or pasted text), integrate it into the wiki. This is the primary way the wiki grows, and it's designed to compound: one source can trigger updates across 5-15 wiki pages, which is normal and desired.
① Capture the raw source:
- URL → use
web_extractto get markdown, save toraw/articles/ - PDF → use
web_extract(handles PDFs), save toraw/papers/ - Pasted text → save to appropriate
raw/subdirectory - Name the file descriptively:
raw/articles/karpathy-llm-wiki-2026.md - Add raw frontmatter (
source_url,ingested,sha256of the body). On re-ingest of the same URL: recompute the sha256, compare to the stored value, skip if identical, flag drift and update if different. This is cheap enough to do on every re-ingest and catches silent source changes.
In practice, this means you can re-run ingestion on a URL without worrying about duplicating work. The hash check is your safety net: if the source hasn't changed, you skip; if it has, you know immediately and can decide whether to update.
② Discuss takeaways with the user, what's interesting, what matters for the domain. (Skip this in automated/cron contexts, proceed directly.)
This step is where you surface the "so what" of a source. In an automated pipeline, you'd skip the conversation and move straight to the next step, but in an interactive session it's a chance to align on what the user cares about before you start writing pages.
③ Check what already exists, search index.md and use search_files to find existing pages for mentioned entities/concepts. This is the difference between a growing wiki and a pile of duplicates.
Before creating anything, see what's already there. The index is your first stop, but for larger wikis you'll want to search the actual files too. This step prevents the classic failure mode of creating a second page for something you already covered.
④ Write or update wiki pages:
- New entities/concepts: Create pages only if they meet the Page Thresholds in SCHEMA.md (2+ source mentions, or central to one source)
- Existing pages: Add new information, update facts, bump
updateddate. When new info contradicts existing content, follow the Update Policy. - Cross-reference: Every new or updated page must link to at least 2 other pages via
[[wikilinks]]. Check that existing pages link back. - Tags: Only use tags from the taxonomy in SCHEMA.md
- Provenance: On pages synthesizing 3+ sources, append
^[raw/articles/source.md]markers to paragraphs whose claims trace to a specific source. - Confidence: For opinion-heavy, fast-moving, or single-source claims, set
confidence: mediumorlowin frontmatter. Don't markhighunless the claim is well-supported across multiple sources.
The page threshold rule is your guard against clutter: a passing mention in one source isn't enough to warrant a page. The cross-reference rule keeps the wiki interconnected, which is what makes it useful for later queries. Provenance markers matter most when you're synthesizing multiple sources, so a reader can trace a claim back to its origin. Confidence is about honesty: if a claim is speculative or only comes from one place, say so in the frontmatter.
⑤ Update navigation:
- Add new pages to
index.mdunder the correct section, alphabetically - Update the "Total pages" count and "Last updated" date in index header
- Append to
log.md:## [YYYY-MM-DD] ingest | Source Title - List every file created or updated in the log entry
Keeping the index and log current is what makes the wiki navigable and auditable. The log entry should list every file you touched, so you can trace what happened on any given day.
⑥ Report what changed, list every file created or updated to the user.
End with a clear summary. The user needs to know what you added or modified, especially if you touched a dozen pages from one source.
2. Query
When the user asks a question about the wiki's domain:
① Read index.md to identify relevant pages. ② For wikis with 100+ pages, also search_files across all .md files for key terms, the index alone may miss relevant content. ③ Read the relevant pages using read_file. ④ Synthesize an answer from the compiled knowledge. Cite the wiki pages you drew from: "Based on [[page-a]] and [[page-b]]..." ⑤ File valuable answers back, if the answer is a substantial comparison, deep dive, or novel synthesis, create a page in queries/ or comparisons/. Don't file trivial lookups, only answers that would be painful to re-derive. ⑥ Update log.md with the query and whether it was filed.
This is the payoff of all the ingestion work. The index gets you started, but for larger wikis you need to search the actual content. When you answer, cite your sources so the user can verify. The filing step is what turns a one-off answer into a reusable resource: if you spent real effort synthesizing, save it so you don't have to do it again. Trivial lookups don't earn a page.
3. Lint
When the user asks to lint, health-check, or audit the wiki:
① Orphan pages: Find pages with no inbound [[wikilinks]] from other pages.
# Use execute_code for this — programmatic scan across all wiki pages
import os, re
from collections import defaultdict
wiki = "<WIKI_PATH>"
# Scan all .md files in entities/, concepts/, comparisons/, queries/
# Extract all [[wikilinks]] — build inbound link map
# Pages with zero inbound links are orphans
This is a programmatic scan, so use execute_code rather than reading files one by one. The comment lines show the intent: scan the relevant directories, extract all wikilinks, and build an inbound map. Orphans are pages that nothing points to, which means they're either new and not yet linked, or they've been forgotten.
② Broken wikilinks: Find [[links]] that point to pages that don't exist.
These are the inverse of orphans: links that go nowhere. They usually happen when a page is renamed or deleted without updating the references.
③ Index completeness: Every wiki page should appear in index.md. Compare the filesystem against index entries.
This catches pages that were created but never added to the index, or index entries that point to files that no longer exist.
④ Frontmatter validation: Every wiki page must have all required fields (title, created, updated, type, tags, sources). Tags must be in the taxonomy.
Missing frontmatter fields break the wiki's structure. Tags outside the taxonomy introduce inconsistency that makes filtering and navigation harder.
⑤ Stale content: Pages whose updated date is >90 days older than the most recent source that mentions the same entities.
If a source has been updated but the page hasn't, the page is likely out of date. This check flags pages that need a refresh.
⑥ Contradictions: Pages on the same topic with conflicting claims. Look for pages that share tags/entities but state different facts. Surface all pages with contested: true or contradictions: frontmatter for user review.
Contradictions are a natural part of a wiki that tracks evolving knowledge. The goal here is to surface them so a human can decide what to do, not to auto-resolve them.
⑦ Quality signals: List pages with confidence: low and any page that cites only a single source but has no confidence field set, these are candidates for either finding corroboration or demoting to confidence: medium.
Low-confidence pages are flagged for a reason: they need either more evidence or a more honest confidence level. Pages with a single source and no confidence field are a similar risk.
⑧ Source drift: For each file in raw/ with a sha256: frontmatter, recompute the hash and flag mismatches. Mismatches indicate the raw file was edited (shouldn't happen, raw/ is immutable) or ingested from a URL that has since changed. Not a hard error, but worth reporting.
This is the same hash check from ingestion, run in reverse. If the hash doesn't match, something changed. Since raw/ is supposed to be immutable, an edit is a red flag; a changed URL is a signal that the source has moved on.
⑨ Page size: Flag pages over 200 lines, candidates for splitting.
Long pages are hard to navigate and maintain. Splitting them into focused sub-pages usually improves the wiki.
⑩ Tag audit: List all tags in use, flag any not in the SCHEMA.md taxonomy.
This keeps the tag system clean. Tags that aren't in the taxonomy are usually typos or ad-hoc additions that should be folded into an existing tag.
⑪ Log rotation: If log.md exceeds 500 entries, rotate it.
A log that grows forever becomes unwieldy. Rotating keeps it manageable.
⑫ Report findings with specific file paths and suggested actions, grouped by severity (broken links > orphans > source drift > contested pages > stale content > style issues).
Severity ordering matters: broken links are the most urgent because they actively break navigation, while style issues are cosmetic. Give the user a clear list with paths and concrete next steps.
⑬ Append to log.md: ## [YYYY-MM-DD] lint | N issues found
Always record the lint run, including how many issues you found. This gives you a history of the wiki's health over time.
Working with the Wiki
Searching
Searching is the most common operation when working with the wiki. You can find pages by content, filename, or tag, and you can check recent activity. The examples below show the typical patterns.
# Find pages by content
search_files "transformer" path="$WIKI" file_glob="*.md"
# Find pages by filename
search_files "*.md" target="files" path="$WIKI"
# Find pages by tag
search_files "tags:.*alignment" path="$WIKI" file_glob="*.md"
# Recent activity
read_file "$WIKI/log.md" offset=<last 20 lines>
Use content search when you remember a concept but not the page name. Filename search is handy when you know the exact page name or want to list all markdown files. Tag search is useful for filtering by topic, especially when tags are consistently applied in frontmatter. The log file gives you a quick view of what changed recently, which is helpful for reviewing agent activity.
Bulk Ingest
When you need to add many sources at once, avoid the trap of processing them one by one. Instead, batch the work:
- Read all sources first
- Identify all entities and concepts across all sources
- Check existing pages for all of them (one search pass, not N)
- Create/update pages in one pass (avoids redundant updates)
- Update index.md once at the end
- Write a single log entry covering the batch
This approach reduces the number of search and write operations, which matters when you are ingesting a large corpus. It also keeps the index and log consistent, since you update them only after all pages are in place. If you process sources individually, you risk repeated updates to the same pages and a cluttered log.
Archiving
When a page is fully superseded or the domain scope changes, archive it rather than deleting it. This preserves history and avoids breaking links unexpectedly. The steps are:
- Create
_archive/directory if it doesn't exist - Move the page to
_archive/with its original path (e.g.,_archive/entities/old-page.md) - Remove from
index.md - Update any pages that linked to it, replace wikilink with plain text + "(archived)"
- Log the archive action
Keeping the original path under _archive/ makes it easy to locate archived content later. Removing the page from index.md ensures it no longer appears in the main index. Updating links prevents broken wikilinks; the "(archived)" marker tells readers the content is no longer current. Logging the action keeps the history complete.
Obsidian Integration
The wiki directory works as an Obsidian vault out of the box:
[[wikilinks]]render as clickable links- Graph View visualizes the knowledge network
- YAML frontmatter powers Dataview queries
- The
raw/assets/folder holds images referenced via![[image.png]]
For best results:
- Set Obsidian's attachment folder to
raw/assets/ - Enable "Wikilinks" in Obsidian settings (usually on by default)
- Install Dataview plugin for queries like
TABLE tags FROM "entities" WHERE contains(tags, "company")
If using the Obsidian skill alongside this one, set OBSIDIAN_VAULT_PATH to the same directory as the wiki path.
This integration is ideal when you want a human-friendly interface to browse and query the wiki. The Graph View gives you a visual map of how concepts connect, and Dataview lets you build dynamic tables and lists from frontmatter. Setting the attachment folder ensures images are stored in the right place, so they render correctly in notes.
Obsidian Headless (servers and headless machines)
On machines without a display, use obsidian-headless instead of the desktop app. It syncs vaults via Obsidian Sync without a GUI, perfect for agents running on servers that write to the wiki while Obsidian desktop reads it on another device.
Setup:
# Requires Node.js 22+
npm install -g obsidian-headless
# Login (requires Obsidian account with Sync subscription)
ob login --email <email> --password '<password>'
# Create a remote vault for the wiki
ob sync-create-remote --name "LLM Wiki"
# Connect the wiki directory to the vault
cd ~/wiki
ob sync-setup --vault "<vault-id>"
# Initial sync
ob sync
# Continuous sync (foreground — use systemd for background)
ob sync --continuous
Continuous background sync via systemd:
# ~/.config/systemd/user/obsidian-wiki-sync.service
[Unit]
Description=Obsidian LLM Wiki Sync
After=network-online.target
Wants=network-online.target
[Service]
ExecStart=/path/to/ob sync --continuous
WorkingDirectory=%h/wiki
Restart=on-failure
RestartSec=10
[Install]
WantedBy=default.target
systemctl --user daemon-reload
systemctl --user enable --now obsidian-wiki-sync
# Enable linger so sync survives logout:
sudo loginctl enable-linger $USER
This lets the agent write to ~/wiki on a server while you browse the same vault in Obsidian on your laptop/phone, changes appear within seconds.
This setup is for environments where you cannot run the desktop Obsidian app, such as a headless server. The agent updates the wiki locally, and obsidian-headless syncs those changes to Obsidian Sync. On your other devices, the Obsidian desktop app picks up the changes automatically. The systemd service keeps the sync running in the background, and enabling linger ensures it continues even after you log out. You will need to replace /path/to/ob with the actual path to the ob binary, and <vault-id> with the ID of the remote vault you created.
Pitfalls
This section collects the traps that most often trip up agents working with Llm Wiki. Treat it as a checklist to run through before, during, and after any operation.
- Never modify files in
raw/, treat every source file as immutable. If a source contains an error, do not edit it; record the correction in the relevant wiki page instead. This keeps the raw archive trustworthy as a historical record. - Always orient first, before doing anything in a new session, read SCHEMA, the index, and the recent log. Skipping this step is the most common cause of duplicate pages and missed cross-references. A few minutes of reading up front saves hours of cleanup later.
- Always update index.md and log.md, these two files are the navigational backbone of the wiki. If you skip updating them, the wiki degrades: pages become hard to find, and the history becomes unreliable. Make updating them part of every operation, not an afterthought.
- Don't create pages for passing mentions, follow the Page Thresholds defined in SCHEMA.md. A name that appears once in a footnote does not warrant its own entity page. If you are unsure whether a mention crosses the threshold, err on the side of not creating a page.
- Don't create pages without cross-references, an isolated page is effectively invisible. Every page must link to at least 2 other pages. If you cannot find two natural links, reconsider whether the page should exist at all.
- Frontmatter is required, every page must include frontmatter. It powers search, filtering, and staleness detection. A page without frontmatter is a liability, not an asset.
- Tags must come from the taxonomy, freeform tags quickly decay into noise, making them useless for filtering. If you need a new tag, add it to SCHEMA.md first, then use it. Never introduce a tag on the fly.
- Keep pages scannable, a wiki page should be readable in 30 seconds. If a page exceeds 200 lines, split it. Move detailed analysis to dedicated deep-dive pages, and keep the main page as a concise overview.
- Ask before mass-updating, if an ingest would touch 10 or more existing pages, confirm the scope with the user first. Mass changes can have unintended consequences, and the user may have context you lack.
- Rotate the log, when log.md exceeds 500 entries, rename it to
log-YYYY.md(using the current year) and start a fresh log. The agent should check the log size during lint, so rotation happens proactively rather than reactively. - Handle contradictions explicitly, never silently overwrite one claim with another. Instead, note both claims with their dates, mark the contradiction in the frontmatter, and flag it for user review. This preserves the history and lets the user decide which claim to trust.
Related Tools
If you prefer a scheduled or command-line driven compile pipeline, llm-wiki-compiler is a Node.js CLI that turns sources into a concept wiki, sharing the same Karpathy-inspired approach. It works with Obsidian, so you can point it at the same vault this skill maintains. The trade-off is that it owns page generation, replacing the agent's judgment on what pages to create, and it is tuned for smaller corpora. Choose this skill when you want agent-in-the-loop curation, where the agent decides what to include and how to structure it. Choose llmwiki when you want a batch compile of a source directory without that interactive judgment.