Every time you start a new conversation with an LLM, it forgets everything. No memory of your preferences, your codebase, your past mistakes, or your project context. You end up repeating yourself — pasting long system prompts, re-explaining your stack, re-establishing constraints.
This isn't a bug. It's a fundamental architectural choice: stateless inference is cheap and parallelizable. But it's increasingly at odds with how developers actually want to use AI tools.
A few different approaches are gaining traction to solve this:
Persistent context windows — Models that maintain state across sessions, either by caching intermediate activations or by using external memory stores. Anthropic's recent work on "artifact memory" and GitHub Copilot's project-level awareness are early examples.
Retrieval-augmented memory — Instead of feeding everything into the context window, systems now index your files, docs, and conversation history into a vector store, then retrieve relevant context on demand. Tools like MemGPT and the emerging RAG-memory hybrids are in this space.
Structured agent memory — AI agents that can read and write to their own persistent memory stores, learning from past actions to improve future ones. OpenAI's recent agent architecture updates hint at this direction.
Here's what the hype glosses over:
Privacy. When your AI remembers everything, where does that data live? On vendor servers? Encrypted at rest? These aren't theoretical concerns — enterprise teams are already running into compliance walls.
Forgetting as a feature. Human memory degrades intentionally — old patterns make way for new ones. A system that remembers everything forever can become brittle, unable to adapt when your stack changes or your team pivots.
Cost. Persistent context isn't free. Caching, retrieval, and storage all add latency and compute cost.
If you're building with AI today, the practical move is to start being intentional about what you ask models to remember: The next wave of developer tools won't just be about prompting better — they'll be about building persistent, intentional relationships with AI systems that actually know your work.
What approach are you using for maintaining context across AI interactions? I've been experimenting with project-scoped memory files and would love to hear what's working for others.