Load less, remember more

Andrey Karpathy posted a gist a few days ago describing a pattern he calls the "LLM Wiki". The idea: a knowledge base where the LLM maintains a persistent, structured wiki between you and the raw sources, instead of re-deriving from scratch at every query. Sounded familiar.

Same shape we've been building on top of CLAUDE.md for our repos. Same shape the Anthropic team uses inside hundreds of internal Claude Code skills.

Karpathy's wiki

The gist outlines three layers. Raw sources sit in an immutable collection. The wiki is a directory of LLM-generated markdown pages: summaries, entity pages, concept pages, cross-references. A schema file (CLAUDE.md, AGENTS.md) tells the LLM how the wiki is structured and what to do when sources arrive or questions get asked.

Three operations bind them.

• Ingest: source comes in → LLM reads it → writes a summary → updates relevant pages → appends a log entry.
• Query: question gets asked → LLM reads the index → finds relevant pages → synthesizes an answer with citations.
• Lint: periodic health check → flags contradictions, stale claims, orphan pages.

Karpathy frames the whole thing against RAG:

the LLM is rediscovering knowledge from scratch on every question. There's no accumulation. Ask a subtle question that requires synthesizing five documents, and the LLM has to find and piece together the relevant fragments every time. Nothing is built up.

The wiki accumulates. It compounds. It persists.

The reason this works, Karpathy notes, is that the tedious part of maintaining a knowledge base isn't the reading or thinking. It's the bookkeeping: updating cross-references, keeping summaries current, noting where new data contradicts old. Humans abandon wikis because the maintenance burden grows faster than the value. LLMs don't get bored and can touch 15 files in one pass. The wiki stays maintained because maintenance becomes near-free.

Map-as-you-go

Karpathy's examples are mostly personal and research-focused: book companion wikis, goal-tracking, weeks-long deep dives on a topic. I'm surprised codebase isn't on the list! That's where we've been applying it at Wisetax: domain rules, cross-repo architecture, API call conventions, per-source ETL pipelines.

A codebase fits the pattern naturally. Tribal knowledge accumulates over years: why one embedding model beat another for our corpus, which legal sources chunk structurally and which by paragraph, what failure mode forced a retry wrapper around a particular API call. Most of it isn't in the code itself. It lives in scattered PR descriptions, Slack threads, Notion tickets, the unwritten lessons we carry in our heads. The wiki/knowledge layer gives it a home.

In practice, most of what we synthesize during sessions does land, with Claude updating knowledge files manually or via hooks throughout. Recently I asked Claude to analyze recent sessions across our repos and flag information we'd forgotten to record in the knowledge layer. One category kept slipping through: the constraint trail, i.e. the alternatives we considered and ruled out, and why. The chosen path's rationale gets filed; the rejected ones often don't (note to myself: open question whether that information is always worth storing).

Meta took the codebase angle too, on a different scale. 4K+ files across four repos and three languages, with AI agents producing code that compiled but was subtly wrong, missing the unwritten conventions each team carried in their heads. They built a pre-compute engine: 50+ agents reading every file in one sweep, producing 59 concise context files (each a digest of a domain's conventions and gotchas), loaded only when the query needs them. The whole pipeline mapped in one go.

I have a feeling Meta's approach might be a little overkill for us (though I'd be curious to run a similar but simplified process). For now, we have a product codebase that ships daily and no appetite for a one-shot mapping pass. Our approach is map-as-you-go. Every new feature/fix leaves the knowledge layer slightly richer than it found it.

The two paths catch different things. A one-shot pass maps the whole codebase, including stretches no one has touched in months. Map-as-you-go only captures what we've actually visited. The rest stays in shadow until we walk through it and light it up. Oldschool strategy-game fog of war, but for code.

Sure. This leaves blind spots. But it self-corrects. The next time someone works in a stretch we haven't mapped, the wiki gets richer there. Plus, it improves and grows on work we'd ship anyway, with no separate mapping effort.

Across time and space

From one example to the next, the starting points differ. Karpathy's wiki for personal knowledge, Anthropic's skills for tooling context, Meta's pre-compute for pipeline conventions, our map-as-you-go for a shipping codebase. Under the surface, the same two mechanisms keep showing up. Progressive disclosure (spatial): a thin router up top, deeper pages loaded on demand. Don't blow the context window when only a slice of the corpus is relevant. Compounding (temporal): don't let the model rediscover what it already learned last session. Persist it, so the next session starts further along.

The two go together. Progressive disclosure makes compounding usable at scale, since the corpus can grow while only the relevant slice loads. Compounding makes progressive disclosure worth building, since otherwise the model rediscovers each session.

Each path wires the two differently. Karpathy uses an index file as router and the wiki as compound. Anthropic uses skill descriptions as router and the skill catalog as compound. Meta uses opt-in references as router and the 59 context files as compound. Ours is a table in CLAUDE.md pointing to each knowledge file with a one-liner.

Four approaches, same shape: load less, remember more. The wiring depends on your tool.