External Memory is important for both people and AI - Part 2 (AI Agents)
Over the last couple months, I’ve been experimenting with not only knowledge bases that I maintain but also knowledge bases agents can maintain. The first one was to help manage our homeschool. I wrote about that here. It’s been helpful in administering our homeschool. One of the simple but time-consuming tasks is creating a daily list of tasks for each child to do each day. This requires:
- pulling information from 5-7 sets of curriculum
- knowing what was done today
- knowing how much time it takes
- creating the list for the next day
I’ve generalized this and built a personal knowledge base of every piece of data I have:
- medical: health and dental
- financial: transactions, investments
- health: Apple Health, Strava, medical labs
- life: Gmail, Google Calendar, Google Drive, Todoist
- past notes: Workflowy
The knowledge base lives in GitHub like the homeschool knowledge base did.
I love being able to ask any questions about my kids’ homeschool, financial, health or life. I can plan for different financial scenarios that help me make better decisions (e.g. the solar one below). I can use it to plan for trips or any other event. Being able to combine data across sources makes this even more useful.
For example, I’ve heard that plugin solar has been legalized in a handful of states. There’s a possibility it will be legalized in California. This means you can have 1,200W of solar and plug it in directly into your house (through a special device) and can directly use the solar generated. I was curious how much of my EV + home electricity use this would cover and what the payback period was. All of this is possible because I have my EV usage data, my Socal Edison electricity data and Claude can do the entire analysis for me: 
It’s also nice being able to take first passes at any content to see if it has any relevance to things I’ve thought about before: 
Being able to ask questions of this data is amazing.
This is a distinct set of files. I think of this as the agent’s memory. The agent has a limited context window it can use which I like to think of as working memory. To persist knowledge across sessions and tasks, the agent needs to offload that. One of the best ways to do that is just plain files (e.g. markdown). That way the agent can easily search for the data it needs given a question. I also don’t care how the agent is organizing its memory files as I rarely review it. This is entirely for the agent to use.
I have since connected Google Calendar and Gmail so the agent is also able to create events on my calendar which has been so helpful to my family. It also has the context for what is coming up on our calendar.
I used this knowledge base to recently help me plan and execute some rollovers. I wanted to consolidate some of my older 401k’s from previous employers and split the Roth dollars into a Roth IRA and then non-Roth dollars into my current 401k. It was fairly technical and tedious. My personal AI assistant (Claude Code + the knowledge base) helped me lay out the steps that were necessary, create the talking points I needed to ask the customer service agents and even helped create tasks for me in Todoist. 
This is the agent’s external memory
In Part 1 I argued that a slipbox works because you put it together yourself. You know where things are and you know how they’re worded, and the act of writing the note is where the thinking happens. I still maintain that. However the knowledge above is agent-maintained. The agent decides how things are structured, what file format to use and where to put things. I rarely go in and edit anything by hand.
I imagine this will continue to get better as both the models and agent harness improve.