axelverne.com / guides / memory
Why does my AI agent forget everything? Memory vs. skills, explained.
It's the most common complaint from new agent owners: you told it your stack, your schedule, your preferences — and three days later it asks again. The fix takes ten minutes once you understand that "forgetting" is actually two different mechanisms, and each one has its own remedy.
The two kinds of forgetting
1. Forgetting within a conversation — the context window
Everything you and the agent say lives in the context window, which has a finite size. Long conversations push early details off the end. When the agent "forgot" something from two hours ago in the same chat, this is why. Symptom: it repeats questions it asked earlier, or contradicts a decision it made earlier in the thread.
The honest remedy is structural, not magical: start fresh conversations for new tasks, and put anything that must survive into persistent memory (below) instead of hoping the thread holds it.
2. Forgetting between conversations — persistent memory
By default, nothing carries over. Each new session starts blank unless the agent wrote facts to persistent memory — a local store it reads at the start of every session (with Hermes, that's ~/.hermes/memories/ on your own machine, no external service).
If you told it something important and it was gone the next day, the blunt truth is: it was never saved. Agents don't reliably auto-save — you have to make saving explicit, or audit what got saved.
The rule that keeps memory useful: facts vs. procedures
Memory rots when it becomes a junk drawer. The distinction that keeps it clean:
- Memory = facts that apply to every session. Who you are, how you like output, what you're building, standing decisions.
- Skills = procedures for how to do a kind of task. They load only when the relevant task comes up, so they cost nothing when idle.
The litmus test from the book: anything a tool call could rediscover in ten seconds doesn't belong in memory. Project file paths, current branch names, this week's priorities — all of that is rediscoverable or temporary. Memory is for the durable stuff.
Three copy-paste fixes
Fix 1 — the memory audit
Run this when the agent starts missing things you know you told it:
Show me everything currently in your persistent memory. For each entry rate it: STILL TRUE / STALE / TOO VAGUE / SHOULD BE A SKILL INSTEAD. Then propose a rewritten memory set that's shorter and higher-signal. Nothing that a tool call could rediscover in ten seconds belongs in there.
Fix 2 — the mistake that never happens twice
When the agent gets something wrong, don't just accept the apology. Every mistake is a missing memory rule:
You just did <X> and it was wrong. Don't just apologise — write a memory rule that prevents it, phrased as a fact about how I work, then show me the rule.
Fix 3 — the weekly decay check
Memory drifts as your projects change. A short recurring job keeps it current:
Review your persistent memory against what you actually did this week. Draft a dated note, and update persistent memory ONLY with facts that will still be true next month. Keep it under 300 words.
Frequently asked
Why does my AI agent forget what I told it?
Two separate mechanisms. Within a conversation, the context window is finite, so older details fall off as the chat grows. Between conversations, nothing carries over unless it was explicitly saved to persistent memory. If it vanished overnight, it was never saved.
What's the difference between memory and skills?
Memory holds facts that apply to every session; skills hold procedures for doing a kind of task and only load when relevant. Facts that apply everywhere → memory. How-to knowledge → skill.
Where is the memory stored?
With a local-first agent like Hermes, on your own machine — ~/.hermes/memories/ by default. You can read, edit, and back it up like any file.
How often should I audit memory?
When the agent starts missing things, and lightly every week or two as projects shift. The audit prompt above takes two minutes to run.
This is one of 58 field-tested prompts in Hermes Agent: 58 Prompts for Power Users — including the full memory-audit sequence and the skills that replace memory abuse.