I built a second brain that actually knows me
A good employee gets better over time.
They learn your business, how you think, what you're working on. Over months, they become a trusted advisor. Think chief of staff.
AI tools are getting there. Most have memory now. But that memory is siloed inside each tool, unportable, and you don't control how it works. Switch surfaces and you're starting from zero again.
I work across a lot of surfaces every day. Primarily Claude Code and Codex in their terminal, desktop apps and phone apps. Each one had its own memory layer. They certainly don't talk to each other.
So I built my own information layer. Here's how I built mine.
🧠 The foundation: a portrait of me, my second brain
Not notes. Not a knowledge base. A set of typed, source-cited facts from the exhaust of my actual work: Claude Code transcripts, Codex sessions, my own journals. No filing. No tagging. No new habits.
It sweeps my chat surfaces every night and pulls out what matters. Six fact types: identity, value, preference, focus, status, relationship.
Each fact carries a confidence score and a source. Some facts are durable, who I am, what I value, how I like to work. Others decay. Everything gets ranked by recency, confidence, and stability. When something changes, the old fact gets superseded, not deleted. The system has a memory of being wrong.
My second brain is accessible via MCP, which lets me flag specific things I want it to remember. One sentence in chat. Done.
Before any of this reaches a model, I approve it. Once approved, my portrait gets injected into the context window of every chat across every surface when it first begins.
📂 The live information layer
On top of the portrait sits everything happening right now.
→ Custom project management software: every conversation knows my most pressing project
→ Custom CRM: relationship context, recent calls, call recordings
→ Email access
These aren't always loaded. They get pulled in when relevant. That's what keeps the context window clean.
The whole thing is model-agnostic. Claude, Codex, Claude Code, whatever I'm working in, they all load the same portrait and reach the same tools.
77% of nightly sweep runs this week produced zero new facts.
A memory system that finds something profound every night is hallucinating. Silence is the success case.
Just like a great employee, it learns over time. Automatically. Effortlessly. I don't manage it. It runs in the background and gets sharper.
Every conversation I have, across every surface, starts with a model that already knows me, knows what I'm working on, and knows what's most pressing. That's what owning your context layer actually unlocks.
And Now you know what our house looks like 😂