Your AI assistant has amnesia: adding a memory layer with mem0

Open a fresh session with your AI assistant and watch yourself type the same briefing you typed yesterday: we use pnpm, not npm. Biome, not ESLint. Tailwind v4, so stop generating a tailwind.config.js. We self-host on Kubernetes, so no, not Vercel. The model nods, produces good work — and forgets every word of it the moment the session ends. Tomorrow you'll say it all again. So will everyone else on the team, each in their own session, each burning the same tokens to teach the same facts to the same model.

That's not a model problem, it's an architecture problem. LLMs are stateless by design; every request stands alone. And the standard workaround — cram more context into every prompt — is the most expensive possible answer to it. The right answer is the same one we reach for in front of a slow ERP backend: stop asking twice. Give the model a memory.

Context windows are not memory

The intuitive fix is a bigger window: paste the repo, the conventions doc, the last three discussions, and let attention sort it out. It doesn't hold up in practice, for three measurable reasons.

You pay for it on every request. Context is metered per call. A 50k-token briefing resent forty times a day is two million tokens spent saying "we use pnpm" — per developer. Memory retrieval injects a few hundred tokens of relevant facts instead; the mem0 team's research paper measures around a 90% token saving against full-context approaches, which matches what our own bills said after we deployed it.

Attention degrades. Models demonstrably lose facts buried in the middle of long contexts. A convention stated on line 40 of a 2,000-line prompt is a convention the model will happily violate — and a violated convention it confidently defends is indistinguishable from a hallucination. Feeding the model less, but targeted context isn't just cheaper; it's more accurate.

The window dies with the session. Whatever the model learned in today's conversation — the bug you chased together, the API quirk you explained, the decision you reached — is gone at the next clear. A context window is working memory. Nobody runs a team on working memory alone.

What a memory layer actually does

mem0 (Apache-2.0, self-hostable) sits between your AI tooling and your LLM and does two jobs. On the write path, it watches conversations and extracts the facts worth keeping — then reconciles each candidate against what's already stored, deciding to add, update, or delete rather than pile up contradictions. The memory that says "we're migrating to Tailwind v4" replaces the one that said "we're on v3"; it doesn't sit next to it. On the read path, a query pulls the handful of memories relevant right now — vector search over the store, scoped to a user, an agent, or a project — and injects them into the prompt.

The core loop is small enough to show whole:

from mem0 import Memory
 
m = Memory.from_config({
    "vector_store": {
        "provider": "pgvector",
        "config": {"dbname": "memories", "host": "postgres"},
    },
    "llm": {"provider": "openai", "config": {"model": "gpt-4o-mini"}},
})
 
m.add("We deploy Next.js standalone images on Kubernetes — never Vercel",
      user_id="jan")
 
hits = m.search("how do we deploy frontends?", user_id="jan")
# → the deploy convention, ranked by relevance, a few hundred tokens

Note the vector store: pgvector. Memories are rows and embeddings, and the Postgres you already run handles both — no new database, no new failure mode. The extraction LLM is pluggable too; a small, cheap model is entirely adequate for deciding whether "we use pnpm" is worth remembering.

If you'd rather not integrate an SDK at all, mem0's OpenMemory server speaks MCP, which is the practically interesting part for development work: Claude Code, Cursor and any other MCP-capable tool can share one memory store. Correct a hallucination once, in any tool, and every tool stops making it.

The numbers, since vibes don't cut it

The mem0 paper benchmarks against the LOCOMO long-conversation dataset, and the shape of the results is what convinced us to run it in production: about 26% higher response accuracy than OpenAI's built-in memory, and — against the paste-everything full-context baseline — roughly 91% lower p95 latency alongside that 90% token saving. Less context, retrieved well, beats more context, dumped raw, on both cost and quality. That's rare enough to act on.

Our own reasons were the unglamorous versions of the same numbers: the token bill for agent workloads dropped visibly the week the memory layer went in, and the class of hallucination we spent the most time correcting — the model confidently reasserting a stack decision we'd reversed months ago — largely disappeared, because the reversal is now a memory that outranks the stale assumption.

Solo dev or team of forty — the benefit compounds either way

For a solo developer, memory is quality-of-life: your assistant finally knows your stack, your style, your project's history, across every session and every tool. The fix-it-once property alone is worth the setup — every recurring correction you've ever typed twice is a memory waiting to be written.

For a team, it quietly becomes infrastructure. Scope memories per developer for preferences, per project for conventions, per agent for operational history, and things start accruing that used to evaporate: the decision and the why behind it, the incident and what fixed it, the API quirk one developer spent an afternoon discovering. A new colleague's assistant knows the house rules on day one — not because someone maintained an onboarding doc, but because the team's daily AI usage kept the memory current. On our custom application projects, where agents handle real slices of implementation work, this is the difference between agents that repeat last month's mistakes and agents that remember making them.

Run it on your own cluster

Everything above works against mem0's hosted platform, but think about what a development memory store contains: your architecture decisions, your unreleased features, your codebase's weak points, fragments of every prompt your team writes. That is intellectual property, and we're not comfortable shipping it to a third party as a background process.

So we don't. mem0 self-hosts cleanly — a Python service, pgvector for storage, an embedding model, all of it deployable as ordinary workloads on the cluster you already run. Memories live next to the databases they describe, under the same backups, the same access control, the same jurisdiction. The hosted platform is the right on-ramp for a quick evaluation; for anything that touches client work, self-hosting is the only version we deploy.

Where to be honest

A memory layer is a cache for facts, and it inherits cache problems. Memories go stale — mem0's reconciliation handles the contradictions it sees, but a decision reversed outside any AI conversation never gets corrected unless you tell it. Skim what's being stored occasionally, the way you'd review any cache policy. Extraction is an LLM judgment call — it occasionally saves trivia and occasionally misses something important; treat it as very good, not infallible. Secrets don't belong in memories — the store is queryable by design, so keep credentials in your secret manager and out of your prompts. And memory is not RAG: it stores facts about you and your project, not your documentation corpus. You may well want both; they answer different questions.

None of that changes the conclusion. Statelessness is the single biggest tax on AI-assisted development, and it's a tax with a published, self-hostable, Apache-licensed fix. Every briefing you re-type is a token bill and an accuracy risk you've already paid once. Stop paying it daily.