Last updated: November 2026
I've been building out an external brain for years, but it has evolved with the rise of a new requirement: I need to be able to share it with an AI agent.
The idea of an external memory system, a "second brain", is that human working memory is small and unreliable, so you build a system outside your head to hold what matters. Niklas Luhmann wrote 90,000 index cards over four decades and published more than seventy books and close to four hundred articles from them. He called it Zettelkasten (slip box), and it was a physical box of numbered notes threaded together by references. Each note was atomic.
The edges, the structure between the notes is where the implicit value lies. Not only the notes themselves, but the graph of relationships between them. Luhmann could walk into his archive, pull a thread, and find connections he had made years ago, likely connections that even he, the author, had forgotten.
In addition to the contents, it remembers the shape of what you've thought. There's also an interesting time component to it for me as well. It can surface a connection between two notes written years apart in a way I couldn't fully recall, but the system preserves.
I started with Obsidian, which made linking notes feel easy and organic. But Obsidian began to feel like a walled garden. Your notes are markdown files on disk (good), but the linking, the graph, the plugins, those are Obsidian-specific.
The move to org-mode was a move toward tool consolidation. As a part of consolidating other tools I use into a professional toolbox, I moved towards Linux and Emacs. Org files are plain text - Emacs org-mode, yes, but also readable in any editor, grep-able, diff-able, version-controllable. The graph lives in a SQLite database that any tool can query. The capture workflow (C-c n c) is a keystroke, instead of a context switch into a different program (consolidating some of the most frequent work into emacs reduces the friction of context-switching, allowing for work to smoothly get done all in the same area. Getting my notes, AI workflows, and coding projects all in one place is really the spirit of this project's changes to org-mode).
Today ~/docs/org/ holds hundreds of org files across 17 subdirectories: computing/, philosophy/, spirituality/, STEM/, economics/, work/, me/, and more. Every node carries a unique ID. Every link is a directed edge in the org-roam graph. The quarterly tracker (~/docs/org/2026-Q3.org) is a living document that captures goals, decisions, and ongoing projects as structured org-mode trees. It's synced between machines via Syncthing, backed up to Backblaze B2 via restic, and available offline everywhere.
The second brain is portable, durable, and mine. No app store, no subscription, no vendor lock-in.
~/docs/org
Org-roam with 500+ nodes is a different kind of Zettelkasten. The links are explicit: an [[id:uuid][Title]] reference says "this note is about that topic" and creates a permanent, searchable edge between two ideas. The quarterly tracker surfaces what's active, and capture (C-c n c) drops a new note into the graph to be digested later.
But it's a human-scale system. I write the notes. I draw the links. The graph knows what I told it to know. The intelligence is all mine; the tool stores and surfaces it.
That's the right design for human thinking and I've the following redesigns aim to make this collection of information useful to an AI agent.
~/projects
Code is a different kind of memory. ~/projects/ holds more than a dozen active repositories. Each one is a formal specification of behavior: tests that encode assumptions, type signatures that encode contracts, commit messages that encode decisions.
This isn't a second brain in the zettelkasten sense. It's an executable memory in a way. The Nix flake in ~/dotfiles is the most extreme version: it is both documentation about my machines and it is the machines. Every package, every service, every config file; declared, reproducible, rebuildable.
An agent can't grep ~/docs/org/: org-mode links, nested outlines, and UUID-based IDs are designed for human cognition, not LLM context windows. And an agent can't navigate ~/projects/ the way I do: it reads individual files, but it doesn't carry a mental model of the dependency graph, the design history, or the rationale behind yesterday's merge.
Hindsight was my first answer to that gap. It stores memory as structured documents (facts, observations, and decisions) retrievable by semantic similarity, tags, or type. Three operations: retain, recall, reflect. The Pi extension (@noctuid/epimetheus, formerly pi-hindsight) and its MCP endpoint put those operations in the agent's hands, so the agent itself decides what to remember and what to look up. Inline hooks used OpenRouter for real-time quality; consolidation and reflection ran against a local Ollama instance.
It worked, up to a point. It proved the need and it taught me that the need is much harder than "store memories and search them."
The first is that every step runs a model. retain, recall and reflect are model calls. That makes memory expensive and non-deterministic, unavailable when the model or the network is, and impossible to audit by arithmetic: you cannot prove what the system believes, only ask it again and hope the answer holds.
Next, Hindsight stores prose, and prose has no per-fact identity. A wrong sentence in a document cannot be pointed at and rejected. You can append a newer note, or summarize the document again, but the wrong thing is still in there, and the next extraction can rediscover it in a stale file or an old transcript and write it back. There is no way to say this value was wrong, and never propose it again.
That second failure has a name now. Most agent-memory systems build the ascent (extraction and retrieval) and almost none complete the descent: correction and forgetting. The Agent Memory Atlas states it as its central finding: retrieval is only half the memory problem. Hindsight had a good ascent and no descent. Memories, the second brain, need updating, refreshing and updating.
unidex is the answer I wanted. It is built around the descent first, and treats retrieval as a solved, pluggable stage.
It is a local, model-free information index: one SQLite file per project, offline, no API key, readable with any client. Three layers:
The kernel contains no model. Nothing in it calls an LLM. Extraction and judgment are the host model's job, routed through skills.
Some important mechanisms surrounding the descent portion:
On top of the belief layer sits procedural memory: a procedure is a node, a transition a claim between two procedures, and ending a session counts the outcome on every transition it traversed. No model judges the run. A step that keeps failing stands out because the failure is counted. Retention runs on three clocks, evidence, tombstones, superseded history.
unidex is reachable two ways: the unidex command, and an MCP surface for hosts with no shell. The engine validates, stores, ranks and refuses.
unidex needed a source of evidence: a way to find the file, the manual page, the note, or the Nix option without knowing its name, and without a language model in the loop. That is ARC.
ARC is a local, offline, config-aware retrieval engine for Emacs, a hard fork of ELISA. It indexes five kinds of source (local files, Emacs Info manuals, org-roam nodes, NixOS module options, and Home-Manager options) and searches them with a hybrid of keyword and vector retrieval: BM25 over SQLite's full-text index, embeddings stored in sqlite-vec, the two fused by reciprocal rank fusion. Results are documents, not chunks, and each carries a source identity and a citation you can jump to.
The retrieval engine needs no chat model. The keyword arm makes no embedding call at all, and answers in milliseconds; the vector arm uses a local embedding model, which is a small thing that turns text into numbers, not a language model. There is deliberately no ask verb: ARC returns evidence, and a caller that wants prose supplies its own model. Emacs gets M-x arc-search; anything outside Emacs gets the arc CLI or the MCP server, both of which reach the same warm database through the running daemon.
Ask it how do I add bookmarks in emacs and it returns the bookmarks node of the Emacs manual, cited, without reading a single line of shell history or sending a byte off the machine. ARC finds the evidence. unidex holds what is believed, and what was rejected. Both run locally, and both keep the model out of the path that decides what is true and what is current.
The stores aren't separate; they're braided:
``` ~/docs/org/ ──→ Syncthing ──→ every machine │ (human-readable, mine) │ ~/projects/ ──→ git ──→ GitHub │ (executable, auditable, shared) │ ARC ──→ SQLite + sqlite-vec ──→ a local index │ (what the machine knows; find anything, no LLM) │ unidex ──→ SQLite ──→ ~/.unidex/<scope>.sqlite (what the agent believes; correctable, no LLM) ⇄ Syncthing + kernel merge ```
ARC indexes the org vault, the project repos, the manuals, and the options as evidence, so the whole machine is searchable by meaning, not just by name. unidex holds the beliefs derived from that evidence, and the corrections: the convention that is actually true, the decision and why it beat the alternative, the value that was wrong and will not be proposed again. The quarterly tracker is the kind of source unidex is built to hold (goals and decisions as claims), with the difference that a wrong claim can now be rejected, and stays rejected.
Syncthing keeps ~/docs/org/ and the unidex SQLite files available on every machine; the kernel's merge reconciles two stores when they meet. Git keeps ~/projects/ versioned. ARC and unidex stay local, one file each, readable with any SQLite client.
The permanent agent brain is not one store. It is four, each with a different job:
| Store | What it holds | For whom | How it remembers |
|---|---|---|---|
~/docs/org/ |
Thoughts, links, quarterly goals | Me | Org-roam graph, explicit [[id:...]] links |
~/projects/ |
Code, config, tests | Me + collaborators | Git history, type systems, CI |
| ARC | The machine's evidence: files, manuals, notes, options | The agent | Hybrid keyword + vector search, citations |
| unidex | Beliefs, decisions, and corrections | The agent | Typed claims, evidence, tombstones |
Hindsight was the right instinct and the wrong foundation: it could remember, and it could not forget on purpose. unidex is the same instinct with the descent built in first. Both run with no language model in the path that decides what is true.
The org vault holds what I think. The projects hold what I build. ARC finds what the machine knows. And unidex remembers what the agent has learned from both, and lets us correct it (so it's not a scrapbook!).