What is AI agent memory?

AI agent memory is the ability for an autonomous agent to store, retrieve, and update knowledge across interactions. Without memory, every conversation starts from zero — the agent has no idea who you are, what you've told it before, or what decisions it already made.

This is the fundamental gap between a chatbot and an agent. Chatbots respond to messages. Agents accumulate knowledge and act on it over time.

Why agents need memory

Large language models have context windows — a fixed number of tokens they can process per call. Once a conversation exceeds that window, older messages are dropped. The model literally forgets.

This creates three problems:

1. Lost context: The agent forgets instructions, preferences, and decisions from earlier in the conversation.
2. Repeated work: Without memory of past results, agents re-derive answers they already found.
3. No learning: Each session is independent. The agent can't improve based on past interactions.

Memory solves this by moving knowledge out of the ephemeral context window and into durable storage that persists across sessions, across conversations, and across time.

How agent memory differs from traditional databases

You might wonder: why not just use a database? Agents can already read and write to Postgres.

The difference is in how the data is organized and retrieved. Traditional databases store structured records — rows in tables with defined schemas. Agent memory stores facts — natural language statements with metadata about scope, relevance, and freshness.

Here's the distinction:

Traditional DB	Agent memory
Schema defined upfront	Facts stored as natural language
Query by exact fields	Query by semantic similarity
Records are immutable until updated	Facts evolve — old versions are superseded
No built-in scope model	Facts scoped by user, session, task
Application manages lifecycle	Memory system manages lifecycle

The key insight: agent memory is a knowledge system, not a data store. It understands that facts change, that some facts are more relevant than others, and that different contexts need different subsets of knowledge.

Key components of an agent memory system

Every agent memory system needs four capabilities:

1. Storage

Where facts live. This can be a vector database, a relational database, a file system, or a combination. The storage layer handles persistence — making sure facts survive across sessions, restarts, and deployments.

The choice of storage backend matters less than you'd think. What matters is the abstraction layer on top: how facts are written, organized, and made available for retrieval.

2. Retrieval

How the agent finds relevant facts when it needs them. There are two main approaches:

• Semantic search: Convert the query and stored facts into embeddings, then find the closest matches by vector similarity. Good for finding conceptually related information even when the exact words differ.
• Keyword search: Full-text search over stored facts. Fast, predictable, and works well when the agent knows exactly what it's looking for.

The best systems combine both — hybrid search that uses semantic similarity for recall and keyword matching for precision.

3. Update

How the system handles changing knowledge. This is where most memory systems fail.

The naive approach is append-only: store every fact, never delete anything. This leads to contradictions — the agent finds both "user prefers dark mode" and "user prefers light mode" and doesn't know which is current.

Better systems implement fact superseding: when a new fact contradicts an old one, the old fact is marked as outdated. It's preserved for audit purposes but excluded from search results. The agent always sees the current state of the world.

4. Scoping

How the system decides which facts are visible in which context. Not all memories should be available everywhere:

• User-scoped facts (preferences, decisions) should persist across all sessions
• Session-scoped facts (current conversation context) should be available only during that session
• Task-scoped facts (intermediate results, scratch work) should be discarded when the task completes
• Agent-scoped facts (learned patterns, capabilities) should be shared across all users

Without scoping, memory becomes a flat pool where everything competes for attention. The agent wastes tokens processing irrelevant facts and risks leaking information between contexts.

Common patterns for agent memory

Vector database approach

Store facts as embeddings in a vector database (Pinecone, Weaviate, Chroma, pgvector). Retrieve by semantic similarity.

Pros: Good semantic matching, scales to millions of facts, mature tooling.

Cons: No built-in scope model, no fact superseding, no lifecycle management. You build everything on top.

Knowledge graph approach

Store facts as nodes and edges in a graph database. Retrieve by traversing relationships.

Pros: Rich relationship modeling, explainable retrieval, good for domain-specific knowledge.

Cons: Complex to build and maintain, requires careful ontology design, harder to integrate with LLMs.

Structured store approach

Store facts in a relational database with explicit metadata — scope, status, timestamps, tags. Retrieve with a combination of semantic search and structured queries.

Pros: Precise scoping, fact superseding, audit trail, works offline. Integrates with existing infrastructure.

Cons: Requires more upfront design. Not as simple as "throw everything in a vector DB."

How db0 approaches agent memory

db0 takes the structured store approach. It gives agents a unified data layer with four scopes (user, agent, session, task), built-in fact superseding, and hybrid search.

Key design decisions:

• Local-first: SQLite by default. No cloud dependency, no API keys, works offline. PostgreSQL available for cross-device sync.
• No LLM required for storage: Facts are stored as-is. No extraction step, no summarization call. This means memory writes are fast and cheap.
• Context assembly: Instead of just retrieving facts, db0 packs the right knowledge into the token budget before each LLM call. Not top-10-and-hope — ranked, budgeted, scoped.
• State recovery: Beyond memory, db0 handles state checkpoints so agents can roll back from bad decisions.

The goal: memory that evolves with the agent, not memory that accumulates until it's useless.

Memory vs RAG

A common question: isn't this just RAG (Retrieval-Augmented Generation)?

RAG and agent memory overlap but serve different purposes. RAG retrieves static documents to ground LLM responses. Agent memory stores and evolves the agent's own knowledge — what it has learned, decided, and been told.

The key differences:

• RAG is read-only: Documents are ingested once. Agent memory is read-write — facts are created, updated, and retired.
• RAG has no scope model: All documents are in one pool. Agent memory scopes facts by user, session, and task.
• RAG doesn't supersede: If a document is updated, you re-index. Agent memory tracks fact evolution explicitly.

Many production systems use both: RAG for external knowledge (docs, APIs, reference data) and agent memory for the agent's learned context (preferences, decisions, conversation history).

For a deeper comparison, see our guide on RAG vs agent memory.

Getting started

If you're building an AI agent that needs to remember things across sessions, you have two choices:

1. Build it yourself: Wire up a vector database, implement scoping and superseding, build a context assembly layer, handle state recovery. This works but takes months and the edge cases are brutal.

2. Use a memory SDK: Tools like db0 give you the full stack — storage, retrieval, update, scoping — in one package. Install, configure scopes, and your agent has persistent memory.

The important thing is to design memory into your agent from the start. Bolting it on later means rearchitecting how your agent handles context, state, and knowledge — a much harder problem than getting it right upfront.

Further reading

• Memory scopes for AI agents — How to design scope hierarchies that prevent information leakage and context pollution.
• RAG vs agent memory — A detailed comparison of retrieval-augmented generation and persistent agent memory.
• db0 documentation — Getting started with db0's memory, state, and context APIs.