Semantic memory search for AI agents

Your AI agent handles a long onboarding conversation. The next day, it asks the same user for their name. That's not a bug. A language model keeps no memory of earlier calls, so without an external memory layer, each request starts fresh and the agent forgets what it just learned. Semantic memory search gives agents durable recall instead, with facts stored outside the model and retrieved by meaning when the app invokes memory search. Below, we cover what semantic memory search is, why keyword search often falls short, and why speed and freshness determine whether memory helps or hurts.

What is semantic memory search?

Semantic memory search is how an agent retrieves facts from a long-term store by meaning rather than exact wording. It has two parts: semantic memory (the store) and the search process that pulls from it when the app needs context.

Start with the store, a long-term collection of facts about the user and the agent's domain, like preferences, prior decisions, and background details worth carrying forward. In the standard reference architecture for language agents, semantic memory is the store of "facts about the world", which in practice means facts about a user and their domain.

Retrieval comes next. To make those facts retrievable by meaning, apps commonly represent them as vector embeddings: numerical representations where two pieces of text with similar meanings get mathematically similar vectors, even when the actual words differ. On the write side, the app extracts a fact from a conversation, embeds it, and stores the vector in a vector database alongside metadata like timestamps and the source message. On the read side, the app embeds the incoming query, runs a similarity search against the stored vectors, and passes the closest matches to the LLM as context.

Redis Iris serves agent context in milliseconds

Redis Iris connects memory, live data, and retrieval in one place.

Try Redis Iris

It's worth being precise about the layers here. The database stores, indexes, and retrieves vectors; the app decides what to remember and when to look it up. This pattern is already common in production, where many AI-native apps use some form of vector-based retrieval before inference to surface context for the agent.

Why AI agents forget without a memory layer

That retrieval layer exists because the model itself doesn't retain context between calls. Unless the app or a provider-managed conversation service persists and resupplies that state, the next call starts without it. An external memory entry, by contrast, can survive across sessions.

Conversation history doesn't persist between calls

The app must supply all session-specific information in the model's current context or let the model retrieve it through tools, so the obvious workaround is to keep re-sending history. That approach runs into three problems as agents scale:

• The ceiling is finite. Context windows are large but finite. When input exceeds the limit, Anthropic's API returns a 400 invalid_request_error rather than trimming gracefully.
• You pay for it every turn. You pay for every previous input token on every request in a chain, so carrying full history gets more expensive the longer the relationship lasts.
• Recall degrades before you hit the limit. Accuracy tends to drop as token count grows. Models also struggle with information buried in the middle of long contexts, and performance degraded 13.9%–85% across one multi-model evaluation as input length grew, even when the models could retrieve every relevant fact.

Stuffing everything into context also invites two related failure modes: context distraction, where a model over-focuses on a long context, and context confusion, where superfluous information drags down response quality. Selective retrieval from an external memory store helps address these problems by aiming to surface a smaller set of relevant facts instead of the whole history.

How semantic search recalls by meaning, not keywords

An external store only helps if the agent can find the right memory when it matters, and that's where keyword search tends to fall short. Lexical ranking methods like Best Matching 25 (BM25) score documents on term overlap, so the same idea expressed in different words can produce zero matches.

Agent memory hits this constantly. A user says "I'm stressed about money" in one session and "worried about finances" weeks later. There's no keyword overlap, yet semantic search maps both phrases to nearby vector positions and retrieves the earlier memory anyway. That doesn't make keyword search obsolete: it still performs better for exact identifiers, product codes, and rare domain terms, which vector embeddings tend to handle poorly. Because each mode covers the other's blind spots, many production systems run hybrid search, combining vector similarity with full-text ranking and metadata filters so no single miss breaks retrieval.

Semantic vs. episodic & procedural memory in AI agents

Semantic memory is one of three long-term memory types agents typically use. They differ in what they preserve and how they shape behavior:

• Semantic memory stores facts, like user preferences and domain knowledge.
• Episodic memory stores experiences, like how the agent solved a problem before, often captured as examples distilled from longer interactions.
• Procedural memory stores instructions: the combination of agent code, model weights, and prompts that determines how the agent behaves.

Which type matters most depends on the agent. Personal assistants often lean on semantic memory for user preferences and profiles, while software engineering agents may also rely on procedural memory like verified code patterns.

Why memory search must be fast & fresh

Knowing what to retrieve is half the problem. The other half is doing it quickly enough, with data current enough, that memory improves the answer instead of degrading it.

Memory retrieval can run on every agent turn

Memory search isn't always an occasional lookup. In systems configured to consult memory on each turn, retrieval runs before one or more LLM calls, and a multi-step agent may retrieve many times per task, so retrieval latency gets multiplied rather than paid once. For many interactive interfaces, 0.1-second responses feel instantaneous, while delays approaching one second start to interrupt a user's flow. Voice agents push the bar further. Some evaluations target sub-200-millisecond latency to support natural turn-taking. The acceptable threshold varies by interface and pipeline.

When retrieval runs on every turn, the data layer underneath it has to keep up, and that's where Redis fits. For many core data operations, it supports sub-millisecond latency: in an Amazon Web Services (AWS) benchmark of a 20-node deployment, Redis reported more than 100 million operations per second at sub-millisecond latency.

Vector search is a separate question, since raw throughput on core operations doesn't predict it. Latency there depends on the index, dataset, recall target, and deployment. On a billion-vector dataset retrieving the top 100 nearest neighbors under 50 concurrent queries, Redis reported 90% precision at about 200 milliseconds median latency, round-trip time included. Redis Search also supports hybrid search: Redis 8.4's FT.HYBRID fuses full-text relevance and vector-similarity scores in one execution plan and can apply metadata filters, so covering the keyword blind spot doesn't mean standing up a second search system. Redis publishes connectors for the major agent frameworks.

Build agents that remember, not agents that guess

Redis Iris gives every agent fresh context and long-term memory.

Try Redis Iris

Why freshness matters as much as speed

Fast retrieval only helps if the memory it surfaces is still true, and staleness is a bigger problem than it looks. Users change jobs, cancel orders, update preferences, and drop old habits, so a fact that was accurate last month may quietly poison this month's response.

Two data points show how easily this slips through. One agent-memory diagnostic found that the top three recalled entries contained old evidence 60.5% of the time, yet the same diagnostic flagged only 3.3% of those stale entries as needing an update. The memory layer surfaced outdated facts and rarely noticed. Models don't catch it reliably either: the best model evaluated on the memory-validity benchmark STALE reached only 55.2% overall accuracy at recognizing when memories were no longer valid, barely better than a coin flip.

The takeaway is that freshness has to be handled at the memory layer, not left to the model to sort out at inference time. And the stakes are concrete: an agent approving a refund needs the customer's current order status, not last week's.

Apps handle this in two ways. They can write memory during a live interaction for immediate availability, or extract it in the background to reduce per-turn latency, accepting a short delay before new facts become available. Redis Agent Memory, one of the managed services in Redis Iris, Redis's real-time context engine for AI agents, handles this with a two-tier design: session memory holds active conversation state at cache speed, and long-term memory persists durable facts as vector embeddings for cross-session retrieval. A background extraction process promotes important information between tiers without blocking the live interaction.

How better memory search improves the user experience

All of this infrastructure work pays off in something users actually feel: not having to start over. The major assistants have shipped persistent-memory or past-chat recall features. ChatGPT surfaces past conversations so users repeat themselves less, Claude searches previous conversations and carries context into new chats, and Gemini recalls past chats for more relevant responses.

How agent memory creates continuity across sessions

The measured gains are starting to show up in production and research. LinkedIn's hiring assistant runs a hierarchical semantic memory system across 1,000+ recruiter seats; in sessions where the assistant invoked memory, the deployment reported a 5–10 percentage-point reduction in negative feedback rate during hiring calibration. One personalization benchmark reported 45% trait incorporation at baseline, which a memory-augmented architecture increased to 75%.

The same similarity machinery can reduce costs, too. Semantic caching can recognize when a new query means the same thing as one already answered and serve the stored response instead of calling the LLM again. Redis LangCache, a managed semantic caching service, reports up to 73% lower inference costs in high-repetition workloads and up to 15x faster responses on cache hits in Redis benchmarks. The same retrieval layer can support continuity for the user and reduce inference costs when cache-hit rates are high.

Fresh context, every call

Redis Iris keeps agent data current so answers stay accurate.

Try Redis Iris

Semantic memory search is infrastructure, not a feature

Semantic memory search helps agents maintain continuity by storing facts outside the model and recalling them by meaning. It works best when retrieval is fast enough whenever invoked and fresh enough to reflect what's true now, not what was true last month.

Redis Iris brings the retrieval side of that problem into one place. Vector search through Redis Search, semantic caching through Redis LangCache, and durable recall through Redis Agent Memory run on the same in-memory platform as your caching and session data, potentially reducing the number of separately operated data systems, depending on deployment. Try Redis Iris to build a memory layer on infrastructure you may already run, or talk to our team about what your agent stack needs.