Category: Programming

Forgetting is Not Always Bad

I was recently reminded of a scene in the sci-fi novel “Sleeping Giants”, where the main character is presented with a fascinating thought experiment about squirrels and how they store food for seasons of scarcity. To paraphrase briefly:

A squirrel gathers far more seeds and nuts than it could possibly consume, hiding this bounty by burying them in seemingly random places. It might stash a thousand nuts or more in a single fall. In spring, it digs, appearing to search at random, in an effort to find enough food to eat.

If a squirrel had perfect memory, we might assume it could gather its food more easily and efficiently. It would remember every single location where it dug and could go directly to each spot to retrieve its treasure. But what would the true outcome be?

The squirrel would starve.

Other animals would have already pillaged some, if not all, of the squirrel’s reserves. Instead of taking cues from its environment, it relies on its perfect memory. In doing so, it often turns up empty-handed more than it gains food. In this case, its limited memory is a true evolutionary advantage.

What Should We Forget?

It’s not that we should simply forget everything. It’s that the things we need to remember may not be in the minute details of what we experience. For the squirrels, maybe it’s more important to recall general areas where nuts are buried, or what the ground looks like when food is stored below. (It’s important to note, that I have no idea if the squirrel-story is accurate, because it came from a fictional book about giant robots, but it’s a great place to help us start thinking.)

Tell yourself a short story about a recent event in your life that involves another person (we’ll call this person “Mark”). Find two other people who do not know Mark, and share the same story with them. After a few days, ask them to recount the story back to you. What details did they remember? What did they leave out?

If I were one of those people, I likely would have forgotten Mark’s name, but I would remember general concepts about where the event took place and what that event was. I might vaguely recall that it happened recently, but I would not remember the exact date and time. However, if you asked me, “Remember that story I told you about my friend, Mark?” I could probably start to recall the event.

Recall

Imagine my brain as a contextual database of fragmented memories. If you asked me, “Do you know my friend, Mark?” how would I search for that information?

I could look for the name “Mark,” but I would be overwhelmed with other unrelated Marks. I might search for “friends of yours,” which could yield a list of individuals mentioned more frequently than Mark. If I searched for “recently mentioned friends,” the list would shorten, and I might deduce that one of the nameless friends must be Mark. I would likely want to confirm that assumption with you. That is how I envision my digital memories functioning.

---

Dual Storage

Take a look at the following table. This is a typical starting point for allowing AI models to work with historical context.

ID	full_memory_text
1	Last week, Mark and I…
2	Yesterday, I went to the store and…

We already have tools to semantically search the full text results (though we may need to break the text into chunks depending on our tooling). This means we can search for terms like “Mark,” “In the past,” or “grocery visit,” and still find the most relevant text for those kinds of queries.

But is there a better way? Absolutely. If we can divide and transform our text into more meaningful pieces based on the types of questions we anticipate, we can enhance both how we store and how we query our memories.

As an important note, this concept is neither new nor original; it is currently being studied, tested, and even packaged into reusable software. If you are interested in reading (or listening to the live poster presentation), there is an excellent article on this subject where the researchers refer to this kind of process as “Graph Neural Prompting”.

Let’s create more human-like memories, where we chunk the experiences (our journal entries) into some sparse but meaningful concepts.

Imagine I took our story with Mark and asked a Large Language Model to generate several summaries around the following themes:

• Who was involved and the subject’s relationship with them?
• What happened during the event?
• What are some general concepts around the event?

Each summary would likely be shorter than the original story (and we can influence this with some straightforward prompting), making them easier to target in specific ways. The hypothesis is that by cutting down the information and focusing it, we are going to reduce hallucinations and improve relevancy in our responses. (I think of this as a cheating form of intentional forgetting!)

Of course, we would still retain the full text, allowing us to search against that as needed. The real magic here is that we can start building a system to query this data for a rich AI conversation.

A Robot Conversation Partner

Last week, OpenAI rolled out the capability to optionally reference past conversations (a very useful “memory” feature). This feature has proven very helpful when answering questions as an assistant.

However, it hasn’t yet achieved that human feel I crave. I wanted to see if I could get closer to “humanity” in my own way. Based on my research into Graph Neural Prompting and my distant undergrad studies in psychology, I created a project with roughly the following flow:

What’s happening here?

1. Whenever we get a new entry from the user, we use an LLM to make a “decision plan”, which is a list of steps to execute in order to provide the most helpful response.
   • Plan options:
     • Look up people
     • Look up events
     • Look up subjects
     • Ask the user for more information
     • Challenge some part of the user’s entry
     • Reflect/mirror/rephrase the user’s entry back to them
     • Affirm/promote the user’s entry
2. For each lookup step, we prompt the LLM to create a statement (or series of statements) to assist us in a semantic search against each concept.
3. We use the LLM to create a contextually relevant context based on the data returned from our searches.
4. We continually add and stack up the contexts of each lookup step to further inform sequential steps.
5. We create individual responses for each of the response/challenge decisions.
6. We combine all of the generated contexts and responses, and we feed them back to the LLM with a prompt to aggregate, condense, and prune until we have a single, cohesive response for the user.
7. We use an LLM to create concept-specific statements for the user’s entry.
8. We create and store embeddings of the user’s concept-specific statements, along with an embedding of the full text, and the full text itself.

The results are promising! After a few conversations, you might find a dialogue that unfolds roughly like this (simulated, since my personal journal is… personal):

AI: What’s on your mind?

Me: Just thinking about my blog post on human-AI interactions…

AI: You talk about that a lot. 🙂 You seem to have developed a passion for working with AI. What’s your post focused on?

Me: It’s about making an AI agent’s memory work more like human memory and striving for more natural conversations.

AI: That sounds like its related to the AI journal project you told me about. Are you looking deeper into some of those theories you mentioned last week? Or do you have a new approach?

Where to go from here?

The power of a system like this is that we can actually do a bit of time traveling and create new, better conceptual embeddings based on older conversations. We can even get into some meta-situations, where we constantly evaluate our past conversations and “reorganize” our memories into new, dynamic concepts.

I’m interested in refining some of this work and in experimenting further with theories that provide models around brain plasticity, short-term vs long-term memory, and also adding time-weighted/temporary attitude to responses based on previous conversations.

TL;DR:

Forgetting (but remembering the important bits) is really important, not so much to save on space (though space is an important factor), but to help us recall things in a natural, useful way.

We can implement a type of forgetting by creating concept-based embeddings that reduce the information stored. Ideally this leads to less hallucinations/irrelevant lookups and more meaningful data retrieval.

We can iterate on these approaches and make our human-machine interactions less mechanical and even evolutionary.

We know we can do these things, but should we?

It’s a question for another day!

Technology around AI changes really quickly, and I’m always looking into my crystal ball, trying to guess at what the future looks like for the tools that I use (and build) that rely on the current big players in the AI space (Meta, OpenAI, Anthropic, etc). I also constantly wonder at the changes that I’m watching, in real time, in tools that I wouldn’t have guessed needed AI.

Sometimes, like in the case of document processing and the analysis of less-structured-data, AI has been an amazing addition to my life. In other areas, like the slow-loading GPT-powered responses that major search engines tried to implement, felt like they got in the way (thankfully, the UX designers have clawed back those implementations to something much more palatable).

Human mimicry on the market?

Beyond data processing, data transformation, and summarization, I have been watching the early attempts at building consistent “personalities” on top of Large Language Models (LLMs). Some of the more interesting types of applications are centered around a type of human-mimicry—more than just achieving human-like writing, but also in emulating emotion in generated speech (in the open-source world, these would be models/apps like Coqui, Tortoise, Bark, and just taking a look at some of the generated samples is enough to demonstrate that emotional speech emulation is not science fiction and will only improve over time).

I consistently see new startups and new apps popping up that make use of the generative capabilities of AI tools and promise to be an “AI Friend”, “AI Girlfriend”, “AI Personal Assistant”, and even an “AI Counselor”. I haven’t searched, but I wouldn’t be surprised to find “AI Pastors” and “AI Life Coaches” already on the app stores.

It still feels like I’m talking to a robot…

Without getting into an endless discussion on whether selling an “AI Pastor” or an “AI Girlfriend” can ever be ethical, I want to focus on the experience of using some of these apps for the user. As a less controversial example, let’s take a look at a journaling app or two that make use of AI personalities.

I tried out one app recently, after it popped up on my feed. Reflectr is a journal app where you make a post that feels like the old versions of a Facebook status update or like a longer-form tweet. Based on your settings (and if you have a paid vs free account), different “personalities” will comment on your post. If you have a premium account, you can even reply to those comments and have a separate conversation with each personality about what you’ve written.

I really wanted to like the app. I know, from studies and life experience, that journaling can be incredibly useful for understanding your own thoughts and for processing life experiences. When I tried the app, I had hoped that the comments and personalities would feel like “wise old friends” or “wise-cracking friends” who would show interest in my post and really push me to (like the app name suggests) reflect on my own personal experiences.

My experience, however, was very sterile.

The personalities like “philosopher” and “optimist” and “comedian” felt like cookie-cutter caricatures of those roles. Their comments were short and lacked depth, and the questions were generic. Within a conversation thread, things would improve as I wrote back and forth, but there didn’t seem to be any connection between conversations with the same personality on different posts.

Ultimately, it didn’t help me accomplish my goals of journaling more or of examining my own thoughts more deeply. I uninstalled the app.

Remember better

I am still fascinated by the concept of what a good AI-infused journal might look like, and the more I experiment and build AI powered apps and agents, the more I am convinced that ability to remember past conversations is a critical feature. But more than that, the way we design these systems to remember and recall is going to be a major differentiator.

Memory as a graph

There are a lot of theories that try to explain the way that humans store and retrieve memories. One of my personal favorite visualizations is to think of small concepts as “nodes” in a 3-dimensional graph, and each node can have dozens, if not thousands of connections (“edges”) to other nodes. Each edge might have a different “strength” which represents how connected two nodes are.

In your own brain, the node for the smell of vanilla might have strong connections to baking cookies or maybe an air freshener you used in your first car. There might also be weaker edges to other nodes, like the weekly coupon book for the grocery store that has a sale on vanilla that you just threw in the recycling bin earlier this morning. And of course, there are secondary and tertiary edges which might be stronger than the weak primary edges—like an association with your spouse (who makes the best cookies) or to your children (who always try to eat the batter or sneak a cookie off the counter when you aren’t looking).

[Im]perfect memories

One of the trademark quirks of our memories (for most of us, with a few exceptions) is that we do not remember things exactly as they were, and that, as we experience more things and remember more things, our past memories are affected. There is a famous word-list study from the 90’s that shows how confidently incorrect we can be at remembering even simple details, and how association can essentially prompt us to remember things that didn’t occur. That study and its tests have been used and cited in many articles and studies since then, and I think its safe to say that the method and measured affect is still relevant.

What about the machines?

When I store a document or text or image in a database, I store the exact data. When I retrieve that data later, it hasn’t changed. In older systems, I started by storing the data with a unique identifier (like an ISBN for a library book). In order to look that item up, I needed to know its identifier. Obviously, some systems needed to support the lookup of items by other attributes (like genre, author, length, etc), and over time the structure we’ve built up in our linear, perfect databases, start to look a little graph-like.

Eventually, you might even develop (and get awarded a patent for) an entire system that excels at finding, categorizing, and presenting what feels like a magic inference of dynamically changing attributes in a way that both humans and machines can understand.

But the retrieval of that information remains relatively perfect (both in repeatability and in the representation of the stored information). That’s great for enterprise software, but my working theory is that this kind of method is too cold for emulating a convincing.

Memory graphs for machines

We already have some excellent tools, readily available to any developer willing to learn and build, that allow us to turn text (words, sentences, paragraphs) into a series of multi-dimensional coordinates that can be used represent how similar or dissimilar words/phrases are as a matter of distance (this is a huge simplification, but I think it keeps the mental model clean while preserving the general nature of vector databases).

These vector databases (and the models used to create the coordinate-like “embeddings” from the documents) are already being used to allow businesses and people to tokenize, store, and semantically search all kinds of textual data. Right now, I could store a journal entry of “eating delicious cookies with my kids”, and then I could semantically search for “smell of vanilla”, and I would probably find the cookie-eating journal entry above entries like “walking to work” or “playing Legos with the kids”.

The searching capabilities are definitely starting to feel more human, and when we let a system take in a user’s prompt, and then we use a semantic search to find (what we think is) relevant information, and we feed all of it into our conversation model, we start to get “memory aware results”. (This is the basic premise of RAG applications). But the responses can have funny results (like remembering the coupon book next more intensely than the cookies) and the overall responses themselves are still not quite warm enough, even with significant modifications to the prompts.

Great. We can remember things, but what next?

I don’t normally think too hard about things I need to remember. I’m one of the lucky ones who can read something yesterday and recall a good chunk of it today (with a funny exception for names and exact dates). I think, however, that I need to have some sort of structure for the memory of my human-like AI personality. I need to make the memory less perfect.

Think about the last really long conversation you had with a friend. What did you talk about? How did you flow from topic to topic? How did you decide what to bring up, when to ask questions, when to share your own tangentially related experiences? How did you know if you could relate (or not relate) to the other experience?

If we can make a rough theory and guess at how we achieved the conversational flow, we can take a craic at creating a system around that theory.

We’ll have to explore that more in a future post, but you can look forward to some rough examples of my memory and conversation models, how I decide what to store (remember) and what to ignore (forget). Eventually, I’ll even describe a few ways of allowing the AI model to make really flexible decisions and how to distill all of the data into something filtered and even a little imperfect (in the best of ways).

The world of software development is in a strange place

On the one hand, we have more processing power, bandwidth, access to knowledge (especially in the form of Open Source Software), and potential than we have ever had. The cost of development (if you are a developer) has been reduced down to the point that time is our most expensive part of building something new. And even now, dozens (if not hundreds) of products exist that promise to reduce the time to an MVP even further. Following popular boards like Product Hunt, Hacker News, or Indie Hackers show that a dizzying amount of products are being released every day. It looks like a sign that we are approaching a time where the people who have problems to be solved can develop the initial solutions themselves.

On the other hand, there are more ultra-generic, copy-pasta applications entering the market—each with their own micro-variation on the same variation. Would-be customers are getting burnt out on the stacking cost of paying for dozens of subscriptions to non-specialized software, only to manually cobble together the output from each tool into something that is actually useful. The same amazing Open Source Software that is available to one company is available to another, and it appears that investors are demanding that startups (and even public companies) have “an AI play”. Outside the occasional home-run, most efforts feel like a half-baked idea, shoe-horned into a product in a way that doesn’t actually help the end user.

Changing for the better …we hope

One of the challenges of the recent explosion of access to AI tooling (LLMs, embeddings, generative AI, etc) is that certain features are starting to be seen as “table-stakes” features when we compare our own products to the products of competitors.

In some ways, this is a welcome change. We have automatic note-taking and transcriptions for meetings, excellent summaries of multiple kinds of documents, the ability to “chat” with developer documentation (though this is still rough), and a number of really amazing accessibility features that would have otherwise been ignored in favor of more money-driven product features.

In other ways, it can be really tiring to have to parse through the well-intentioned, but absolutely flawed output of an “AI” chat agent, which is clearly unaware of the information you are seeking (but does not offer another avenue to find what you are looking for). The legal challenges of letting a machine talk to your customers has already seen some fairly large court cases in the United States (which, compared to much of Europe, has a much more relaxed regulatory environment).

My hopes for future software products

Even with my cynicism toward overly generic AI software and poorly placed AI “features,” I still see a place for AI tooling in many (if not all) industries.

I find myself in the camp of people who think that in the near-future, AI can alleviate a lot of busy-work in terms of data-entry, data-conversion, and data-validation. The power of AI to save people time should be, in my opinion, the primary focus of most new AI powered software. And, if we (as creators of software) are clever about it, this should lead to more consistent experiences for our customers, and more time for us to understand the problems we are trying to solve.

Beyond time-savings, I have already seen the beginnings of the benefits of generative AI in terms of writing software. I am convinced that, without the recent advancements in AI, I would have had to make significant trade-offs in my own product offerings. I have watched, with great interest, the development of “AI App Generators” that promise to create an MVP of your product with only a few prompts from a single user. I have installed and trialed several interface-building systems that have similar promises. While these efforts are incredibly impressive (some are even a little useful), the promise has yet to be fulfilled.

I have several theories on what it would take to move these AI application generators past their infancy and into a more robust form, but in the near-term, I’m already sold on existing patterns for getting AI to write pieces of software that can be reasonably composed together. It’s clear that there a lot of different approaches to AI-powered programming (from co-pilots to prompt-libraries), but to me, what matters the most is the consistency of the output. I’ve been organizing my own strategies and approaches, and I’m testing them in my own production applications.