LLM-Maintained Knowledge Base

Complete Setup Guide for Andrej Karpathy's proposed setup

Andrej Karpathy recently proposed LLM Knowledge Bases, a system where an LLM acts as a librarian — ingesting raw sources, compiling a structured wiki, and answering complex queries against it. You rarely edit the wiki directly; the LLM owns it. Here is a quick setup guide to test this setup for yourself.

What we are setting up?

The core loop is: ingest raw → compile wiki → query & enhance → repeat
Obsidian is the viewer, the LLM is the writer
Index files (_master-index.md, per-topic _index.md) replace the need for RAG at small-to-medium scale (~100 articles, ~400K words)
Every query result can be filed back into the wiki, so exploration compounds over time
CLAUDE.md is the instruction set that persists the LLM’s behavior across sessions

Architecture Overview

vault/
├── CLAUDE.md              # LLM instructions (the "constitution")
├── raw/                   # Inbox — dump source material here
│   └── _processed/        # Compiled files move here
├── wiki/                  # LLM-maintained knowledge base
│   ├── _master-index.md   # Top-level index of all topics
│   └── <topic>/           # One folder per topic
│       ├── _index.md      # Lists all articles in this topic
│       ├── article.md     # Individual wiki article
│       └── assets/        # Images for this topic
├── output/                # Query results, reports, slides
└── .obsidian/             # Obsidian config and plugins

Step-by-Step Setup

Step 1: Create the Vault Structure

Open a terminal and create the directory skeleton:

mkdir -p vault/{raw/_processed,wiki,output}

Step 2: Initialize Obsidian

Open Obsidian → “Open folder as vault” → select the vault/ directory
This creates the .obsidian/ config folder automatically

Step 3: Install Obsidian Plugins

Community plugins (Settings → Community plugins → Browse):

Plugin Purpose Local Images Plus Downloads remote images to local when pasting URLs or clipping articles Marp Slides Renders Marp-format markdown as slide decks Dataview (optional) Query your wiki with SQL-like syntax inside Obsidian

Browser extension:

Extension Purpose Obsidian Web Clipper Clips web articles as .md files directly into raw/

Configure Web Clipper’s default save location to raw/ so clipped content lands in the inbox automatically.

Core plugins to enable (Settings → Core plugins):

Backlinks
Outgoing links
Graph view
Tags
Page preview

Step 4: Create the Master Index

Create wiki/_master-index.md:

# Knowledge Base Index

Topics will be listed here as they are created.

This is the entry point the LLM reads first when navigating the wiki.

Step 5: Write CLAUDE.md

Create CLAUDE.md in the vault root. This is the instruction file that tells the LLM how to behave across every session.

Prompt: Core Rules

# Vault - LLM-Maintained Knowledge Base

## Knowledge Base Rules

- This is an LLM-maintained knowledge base. You are the librarian.
- The wiki/ folder is YOUR domain - you write and maintain everything in it.
I rarely edit wiki files directly.
- raw/ is the inbox. When I dump files here, you process them into the wiki
during a "compile" step.
- wiki/_master-index.md is the entry point. It lists every topic folder with
a one-line description. Always keep this up to date.
- Each topic gets its own subfolder in wiki/ (e.g., wiki/ai-agents/) with its
own _index.md that lists all articles in that topic with brief descriptions.
- Always use [[wiki links]] to connect related concepts across topics.

Prompt: Compile Workflow

## Compiling Raw Material

When compiling raw material:
1. Read the raw file
2. Decide which topic it belongs to (or create a new one)
3. Write a wiki article with key takeaways and relevant links
4. Update that topic's _index.md
5. Update wiki/_master-index.md
6. If a raw file spans multiple topics, create articles in both and cross-link
7. Move the compiled file to raw/_processed/

- Keep articles concise - bullet points over paragraphs.
- Include a ## Key Takeaways section in every wiki article.

Prompt: Compiled File Tracking

## Compiled File Tracking

- After compiling a raw file, move it to raw/_processed/.
- This makes it easy to see what's pending (raw/) vs already compiled
(raw/_processed/).
- If a raw file is updated after processing, move it back to raw/ for
recompilation.

Prompt: Image Handling

## Image Handling

- When a raw file or article references images, store them in
wiki/<topic>/assets/.
- Reference images using relative paths: ![description](assets/image.png).
- The obsidian-local-images-plus plugin handles pasted images in Obsidian,
but when compiling raw files with images, manually copy them to the
appropriate assets/ folder.

Prompt: Commands

## Commands

- "compile" — Process everything in raw/ that hasn't been compiled yet into
the wiki. Move compiled files to raw/_processed/.
- "audit" or "lint" — Review the wiki for inconsistencies, broken links,
gaps, and suggest improvements.
- "query: [question]" — Research the question across the wiki and write the
answer to output/ as a markdown file.

Prompt: Q&A Workflow

## Q&A Workflow

- When answering questions, read _master-index.md first to navigate, then
drill into the relevant topic _index.md, then read specific articles.
- Write answers to output/ as markdown files so they're viewable in Obsidian.
- For visual outputs, generate the appropriate format:

Slides — Use Marp-compatible markdown (frontmatter: marp: true)
Diagrams — Use Mermaid code blocks (rendered natively in Obsidian)
Charts/images — Generate matplotlib/other scripts and save output PNGs
to output/

Prompt: Filing Outputs Back

## Filing Outputs Back into Wiki

- When a query result is substantial enough to be a standalone article,
file it into the wiki:

Move or adapt the output into the appropriate wiki/<topic>/ folder
Update the topic's _index.md
Update wiki/_master-index.md
Add [[wiki links]] to connect it with existing articles
- This feedback loop ensures explorations and queries always "add up"
in the knowledge base.

Step 6: Set Up the Ingest Pipeline

For web articles:

Install the Obsidian Web Clipper browser extension
Configure it to save clipped articles to raw/
Enable “Download images” in the clipper settings or use Local Images Plus to pull them after clipping

For papers/PDFs:

Save PDFs to raw/ — Claude Code can read PDFs directly during compile

For code repos/datasets:

Clone or save relevant files to raw/<source-name>/
The LLM will extract key concepts during compilation

Hotkey tip: Set up a keyboard shortcut to download all images on a page locally, so the LLM can reference them without network access.

Step 7: First Compile

Drop some source material into raw/, then tell the LLM:

compile

It will process each file, create topic folders, write articles, build indexes, and move processed files to raw/_processed/.

Step 8: Query the Wiki

Once you have enough compiled articles, ask questions:

query: What are the main differences between X and Y?
query: Summarize everything we know about Z
query: Generate a slide deck on topic W

Results land in output/ and are viewable in Obsidian.

Step 9: Audit Regularly

Periodically run:

audit

The LLM will:

Find broken [[wiki links]]
Spot inconsistent data across articles
Identify gaps and suggest new articles
Flag stale content
Use web search to impute missing data when appropriate

Usage Patterns

The Core Loop

Ingest → Compile → Query → File back → Repeat

Ingest: Clip articles, save papers, dump notes into raw/
Compile: LLM processes raw into structured wiki articles
Query: Ask complex questions, get answers in output/
File back: Promote valuable outputs into the wiki
Repeat: The knowledge base compounds with every cycle