The memory system behind an Agentic OS
Mike Codeur
You use Claude Code every day. But on every new project, you end up re-explaining the same things: conventions, commands, decisions, mistakes to avoid.
And when people talk about fixing that, everyone's looking for "the best AI memory". Wrong question.
There isn't ONE memory — there are three different problems: where to store the memory, how the agent retrieves the right info, and how that memory gets cleaned up over time.
In this article, I break down the 6 levels of memory for Claude Code, and more importantly how to organize them into 3 categories so you can pick the right setup without piling up tools at random.
The real problem: dispersion, not forgetting
Claude Code already has basic memory today:
CLAUDE.md— a static file that provides rules and context at startup- Auto Memory — the system where Claude writes its own notes into a
/memoryfolder
So yes, Claude is starting to remember. But once your setup gets serious, the problem changes. The real issue is no longer "does Claude forget?" — it's dispersion.
You have:
- Rules in
CLAUDE.md - Auto notes in
/memory - Decisions in Obsidian
- History in conversations
- Context in the repo
…and sometimes none of these tell exactly the same story.
That's exactly the problem I have on my own setup: Claude Code, OpenClaw, Obsidian, YouTube scripts, research, business decisions. If I don't structure the memory, the agent can find pieces of information — but not necessarily the right source of truth.
The 3 layers of agentic memory
Good agentic memory isn't a single tool. It answers three distinct questions:
| Layer | Question | Levels |
|---|---|---|
| Storage | Where does the memory live? | 0, 1, 2 |
| Retrieval | How does the agent find the right info? | 3, 4 |
| Consolidation | How does the memory get cleaned / matured? | 5 |
Here's the pitch: first pick the category you need, then the right level. Not the other way around.
STORAGE — Where does the memory live?
Level 0 — CLAUDE.md, the static memory
The baseline. A Markdown file loaded at every Claude Code startup, containing conventions, commands, architecture, project rules.
- Robust because it's simple
- But static: if no one edits it, it doesn't improve
- When it's enough: short solo project, 1 repo, 1 context
For many projects, this is already enough. I actually think 60% of devs should start here, not with a vector database or a complicated memory tool.
Level 1 — Auto Memory: Claude takes its own notes
The native Claude Code system (since v2.1.59+). Claude writes itself into ~/.claude/projects/<project>/memory/:
MEMORY.mdas index- Typed files (
user.md,feedback.md,project.md,reference.md) - Topic files read on demand
Important limits:
- Only the first 200 lines of
MEMORY.mdor 25 KB are loaded at startup - Auto Memory is machine-local — no cross-device sync
- All worktrees of the same repo share the same folder
It's a real step forward: Claude learns from your corrections. But it's still local, simple, file-based memory.
Level 2 — Obsidian (or external vault): your real long-term brain
⭐ My signature pattern. If you already have an external brain (Obsidian, Notion, a vault), why create a second one?
Pattern: Claude Code reads/writes directly into ~/repo/obsidian-mikecodeur/.
Key advantages:
- Visible by humans AND AI (graph view, backlinks)
- Versioned with Git
- Cross-machine automatically
- Shareable with others
- Single source of truth for content, decisions, research
That's exactly what I do for this blog, my videos, my research. Claude writes drafts, updates notes, references with [[backlinks]].
RETRIEVAL — How to find the right info?
Level 3 — RAG (retrieval over codebase + docs)
Index your code, your docs, your notes into embeddings + similarity search.
Difference vs memory: no "learned fact" — it's an index over existing content.
Concrete tools:
claude-contextMCP — embeddings + retrieval for Claude Coderepomix+ embeddings on the codebase- Cursor codebase index — market reference
- Cody (Sourcegraph) — enterprise RAG
aidersemantic add
When it's worth it: codebase > 100k LOC, or massive internal docs. Before that → over-engineering.
"Find me the file that handles auth" on a large monorepo, without giving the path → that's the perfect RAG use case.
Level 4 — Dedicated semantic memory
Retrieval, but for facts/preferences, not for source content.
Tools:
claude-mem(github.com/thedotmack/claude-mem) — Claude Code plugin for persistent memory compression- Memory tool API (Anthropic, shipped with Sonnet 4.5) — native primitive to store/retrieve via tool calls
- mem0, Letta, Cipher — open source alternatives
Difference vs Level 3: you retrieve conclusions (facts, feedback), not source content.
RAG retrieves documents. Semantic memory retrieves conclusions: what the agent has learned.
claude-mem works via Claude Code hooks (SessionStart, UserPromptSubmit, PostToolUse, SessionEnd) + a local worker + SQLite/FTS5 + an optional vector index. It's not just an MCP server — it's a full memory pipeline.
CONSOLIDATION — How does memory mature?
Level 5 — Dream / sleep-time compute
Concept: while you're not working, the agent re-processes its memory. It re-reads its traces, summarizes, merges, deletes noise, transforms history into knowledge.
Implementations:
- OpenClaw dreaming —
light/deep/REMphases, redacted transcripts, Memory Palace Diary - Letta sleep-time agents — secondary agent that consolidates during idle time
- memory-wiki + Imported Insights (OpenClaw) — ChatGPT ingestion + consolidation
When it's worth it: long-running agents that accumulate noise, need to "refine" the memory. Overkill if you have low volume.
At this level, memory is no longer just stored or retrieved. It's reworked.
Which memory should you choose?
The trap is to pile up all the tools thinking that more = better. The right strategy is to pick the level matching your use case.
| Profile | Recommendation |
|---|---|
| Small project, solo dev | CLAUDE.md (Level 0) is plenty |
| Regular Claude Code, multi-project | Enable Auto Memory (Level 1) |
| Creator / long-term consultant | External Obsidian (Level 2) ⭐ |
| Large codebase / monorepo | Add RAG (Level 3) |
| Multi-session autonomous agent | Semantic memory (Level 4) |
| 24/7 long-running agent | Consolidation / dreaming (Level 5) |
Conclusion — Memory is the OS of your agents
Memory is what turns Claude Code from a tool into a system that learns from you. It's exactly the foundation of a personal Agentic OS: without structured memory, you start from scratch on every conversation.
The goal isn't the most complicated stack. It's a memory that genuinely helps your agent work better without creating a Rube Goldberg machine.
🎥 Watch the full video: The memory system behind an Agentic OS (16 min)
📥 Lead magnet — The agentic memory pack (free): mkc.sh/agentic-os-memory — templates, Obsidian patterns, copy-paste prompts
📬 The Agentic Dev newsletter: every Tuesday, an agentic stack broken down. Sign up