- Robin van Veen - AI Automation
- Posts
- Your AI agents keep forgetting. Here's the fix.
Your AI agents keep forgetting. Here's the fix.
The tool will change. Make sure your knowledge doesn't.
My entire business runs from one app. It's not what you'd expect.
Not a CRM. Not a project management tool. Obsidian.
Over the past two years I've rebuilt my AI agents multiple times. New models. Different tools. Once because a platform just disappeared overnight. Each time I rebuilt, the agents changed. The knowledge in Obsidian didn't move.
That's the thing most people miss when they set up AI agents: where does the knowledge live?
If it lives inside the tool, you're renting your own context. Every migration costs you weeks. Every platform shutdown takes a version of your business with it.
The problem with tools as your foundation
Most businesses treat the tool as the foundation. The AI, the CRM, the automation platform, whatever it is. They build on top of it. When the tool changes or shuts down, everything built on top resets.
The newest tool is genuinely better. I get the appeal. But better is relative. Today's best model gets replaced in six months. Today's best platform raises prices or pivots or folds. If your knowledge lives inside it, you're starting from zero when that happens.
The other way to build
Knowledge is the foundation. Tools plug into it.
My agents read from Obsidian before they do anything. They write back to it after every conversation. Every decision, every client detail, every process that worked, it gets captured. The next time the agent runs, it starts from that compounded base instead of a blank slate.
Two years in, the agents know more than they did at the start. Not because the models improved. Because the knowledge compounded.
When a better model comes out, I connect it to the same base. When a platform shifts, the connection moves. The knowledge doesn't.
How to connect Obsidian to your LLM
You don't need to be technical to set this up. Here's the approach.
Step 1: Structure your vault around your business
Create one file per area of your business. Mine looks like this:
Business Profile: company overview, products, what we've delivered
Sales: active prospects, common objections, what's worked
Operations: processes, client notes, how we do things
Finance: monthly numbers, targets, what's trending
Marketing: voice guidelines, content themes, what to avoid
Each file stays short and scannable. Clear headings, short paragraphs. The AI needs to find the relevant section fast, not read an essay.
Step 2: Tell your AI where to look
Before any task, paste the relevant note into your conversation, or connect your AI tool to your vault directly (tools like Claude and ChatGPT support this). Either way, the principle is the same: before starting work, the agent reads the context. Not because you reminded it, but because that's the workflow.
I have a simple rule: sales task, read the sales file. Marketing task, read the marketing file. Client work, read the operations notes for that client.
Two seconds of setup. No more repeating yourself.
Step 3: Write back after every session
This is the step most people skip. At the end of a session, have the agent summarize anything worth keeping and update the relevant file. New client detail, a decision you made, a process that worked or didn't.
You can do it manually with a simple prompt: "what from this conversation should I save to my notes?" Or automate it if you want. Either works.
The write-back is what makes the knowledge compound. Without it, you're still starting from zero every time.
Step 4: Keep the vault honest
Add to it when something changes. Fix the process that didn't work. Update the client notes. The more accurately it reflects your actual business, the more useful the agent becomes.
One hour of vault maintenance a month saves hours of repeated context-setting across every AI conversation you have.
What this looks like in practice
When I open a client conversation now, my agent already has the context. It knows the industry, what we've done before, what didn't work last time. No briefing needed.
When I write a proposal, it pulls from real case studies I've added over time. When I do a finance review, it reads last month's notes and builds on them.
The agent isn't useful because the model is good. It's useful because the knowledge base is good.
The honest cost
Setting this up takes a few hours. Keeping the vault current takes maybe 20 minutes a week if you're consistent. The write-back habit is the hardest part because it takes discipline to do it at the end of a long session.
But the alternative is rebuilding context every time something changes. And something always changes.
Build around the knowledge. Make the tools interchangeable. The tools will keep getting replaced. This doesn't have to.
Robin
P.S. Want me or my team to help you set this up? Book a short call here and we'll see where we can help.