Home Blog Knowledge
Knowledge

AI agent memory: what to remember, for how long, and what to never store

Memory makes an agent feel smarter, and it also makes it a store of data you now have to manage. A practical way to decide what an agent should remember.

By The Cadrix team6 min read
A wall of old card-catalogue drawers with brass handles and labels
Photo by MIKE STOLL on Unsplash

People use the word "memory" for three different things, and mixing them up is where most design mistakes start. Before you give an agent memory, decide which kind you actually need.

Three kinds of memory#

The conversation. What was said earlier in this chat or run. It lives in the model's context window and disappears when the run ends, unless you save it. This is short-term memory.

Notes the agent keeps. Facts, preferences or progress written to storage outside the context window and loaded again later. Anthropic's engineering team calls this structured note-taking in its guide to context engineering, and describes it as a way for an agent to track progress across long tasks. This is long-term memory in the sense that matters for workflows.

Shared knowledge. Documents and records that many runs read but no run writes: policies, product manuals, price lists. That is a knowledge base, not memory, and it should be managed as one.

Most business workflows need the third kind and a little of the first. Long-term memory is the one to add last and add carefully.

Do you need long-term memory at all?#

Ask what would go wrong without it. If the answer is "the agent would ask the customer to repeat their order number", pass the order number in from the trigger. If it is "the agent would not know this customer prefers email follow-ups", store that as a field on the customer record you already have, not as an agent's private note.

A structured record has an owner, a format and a way to correct it. A free-text note an agent wrote to itself has none of those. In our experience the cases where an agent-written memory really helps are narrow: long-running tasks that span many sessions, and personal assistants that adapt to one user.

Where it can live#

Anthropic's documentation for its memory tool describes a simple pattern: memory is a directory of files that the model reads and writes through tool calls, and your application decides where the files really live. The point to take from that design is that memory is client-side storage you control. You choose where it is stored, how long it is kept and who can read it.

In a workflow tool, the same options exist in different shapes:

  • Chat history for a session, so a follow-up question has context.
  • A knowledge base for documents everyone should share, with sources you can see.
  • Variables and flow state for values that must carry from one step to the next within a run.
  • An external store you already run, such as a spreadsheet or a database row, written and read by workflow steps.

In Cadrix, the last option is often the best one for anything that is a business fact. A step writes a clearly named field to a Google Sheet or calls your own system over a web request, and a later run reads it. Anyone can see it and fix it.

What to never store#

Write this list down before the first agent writes anything:

  • Secrets. Passwords, API keys, tokens and card numbers do not belong in a memory file. Credentials live in encrypted connections, not in notes.
  • Identifiers you do not need. Government IDs, health details and anything else sensitive, unless the process truly requires it and you have decided who may see it.
  • Inferences about people. A note that says "this customer seems angry and likely to churn" is a judgement stored with no record of who made it or why.
  • Instructions. If a document or an email can cause the agent to write a "note" that later changes its behaviour, you have created a way for untrusted text to plant standing orders. Treat anything written from external input as untrusted when it is read back.

Anthropic's memory tool documentation lists the same concerns for implementers: sensitive information, file size, expiry and path traversal. It says the model usually refuses to write sensitive information to memory, but advises adding your own validation for stronger guarantees. Do not rely on the model alone.

Expire it#

The GDPR's principle of storage limitation says identifiable data should be kept for no longer than necessary for its purpose, and data minimisation says it should be limited to what is necessary. Whether the GDPR applies to you is a legal question, but the discipline is useful everywhere. Memory that never expires becomes a pile of stale facts that the agent treats as true.

Practical rules:

  1. Give every kind of note a purpose in one sentence.
  2. Give it a time to live. A preference might last a year, a task note a week.
  3. Store where it came from and when, so a person can judge whether it is still true.
  4. Have a way to delete one person's memory on request, which is easier if it is keyed by a customer ID instead of buried in free text.
  5. Cap the size. Anthropic's guidance also suggests tracking file sizes and limiting how much a single read returns.

Check what the agent actually used#

Memory is invisible unless you look. Make a habit of opening the run trace for a few runs and reading what was loaded into the step, including anything retrieved from memory or a knowledge base. If a surprising fact appears in an answer, you want to be able to find where it came from in a minute.

A small pattern for Cadrix#

For a support assistant that should remember a customer's stated preferences:

  1. The trigger passes the customer ID and the message.
  2. A step reads that customer's row from your sheet or system, via a Google Sheets tool or a web request.
  3. The agent answers, with the preference fields included in its instructions.
  4. If the customer states a new preference, the agent proposes an update as structured output.
  5. A Condition checks the update against an allow-list of fields. Anything outside it is dropped.
  6. Optionally, a Human Input step lets a person confirm updates that touch contact details or consent.
  7. A final step writes the field with a timestamp.

Where the human goes. Approve writes, not reads. A person should confirm any memory update that changes contact details, consent or anything that will drive a future action. Low-stakes preferences can be written automatically, as long as someone reviews a sample every month.

Sources#

Get new posts by email

Practical notes on governed AI workflows, in your inbox. Unsubscribe whenever you like.

Opens your email app to send the request. We use your address only to send the blog. See the privacy policy.

Keep reading