Continuous Learning Won't Come From the Weights

DeepSeek's Liang Wenfeng told investors that continuous learning is the missing piece on the road to AGI, arguing that today's frozen models cannot learn over time. Tolaria, an open-source knowledge base, exemplifies a files-first approach that stores learning in durable, inspectable, portable files rather than model weights.

In a rare investor briefing, DeepSeek's Liang Wenfeng was asked what's actually missing on the road to AGI. His answer wasn't a bigger model, or video, or another modality. He was blunt: the thing today's models can't do is learn continuously . Solve that, he said, and you reach a gradual singularity — and eventually embodied intelligence. He's right. And it's worth sitting with how many people, from very different corners, now point at the same wall. Karpathy talks about agents that install skills like .md files. OpenAI shipped Dreaming — background memory synthesis between sessions. Microsoft built agent identity into the OS at Build. openclaw added /dreaming . The frontier and the ecosystem have converged on the same missing piece. Here's the uncomfortable part for anyone who assumes the fix is "just train more": the frozen model can't be where continuous learning happens. Three reasons, and none of them are going away: Economics. You cannot retrain a model per user, per session. The whole appeal of a foundation model is that it's shared. The moment you want it to remember your last conversation, you've left the regime that training operates in. Opacity. Weights are a black box. You can't open them up and see what the model "learned" last Tuesday. You can't delete one wrong fact. You can't audit what it retained about you. For anything that touches trust — a coworker, a support agent, a system of record — that's disqualifying. Lock-in. Weights belong to one vendor's model. If your agent's accumulated experience lives inside a specific set of parameters, it dies the day you switch models. And you will switch models — the field moves too fast to marry one. Put simply: continuous learning that lives in the weights is continuous forgetting from the user's side. Every new session starts from zero. If the learning can't live in the model, it has to live beside it — in a store that's durable, inspectable, portable, and model-agnostic. Increasingly, that store looks like plain files. Tolaria, an open-source knowledge base that surfaced this week, is a clean example of the pattern: markdown files, a git repository, YAML frontmatter, an MCP server, and AGENTS files so external AI tools can read and write the vault. Its design principles read like a manifesto — files-first, git-first, offline-first, zero lock-in. No accounts. No cloud dependency. Your knowledge is just files you own. That's not a coincidence. It's where the whole "files-first" movement is going, because files solve exactly the three problems the weights can't: they're portable any editor, any model , inspectable you can read and diff every change , and versioned git already solved memory-over-time decades ago . Our own agent memory is built this way — plain markdown, git-synced, human-auditable. We put it to a hard test this month: we rebased our agent runtime across four months of upstream changes and swapped the underlying inference model entirely. The runtime changed. The model changed. The agent's memory came through intact, and it still cited what it had learned weeks earlier, under a different model. That is the entire point. The learning has to survive the model. Here's where most of the "just use markdown" takes stop too early. Writing files is the easy part. A folder of notes is not continuous learning — it's a filing cabinet. The hard, valuable work is the runtime around the files: This runtime is the layer we call Soul Memory. The markdown store is a commodity — anyone can write files. The cognition layer on top is the moat. If you take one thing from our experiential memory research https://blog.clawsouls.ai/posts/experiential-memory-paper/ , take this: not all memory helps. In a blind study, agents given synthetic memory — tidy, summarized recaps — scored below agents given no memory at all. A confident, lossy summary turned out to be worse than an honest blank, because it made the agent overconfident about things it had subtly gotten wrong. That is the trap in naive continuous learning. When you compress experience into a neat summary, the first things dropped are the caveats — this was uncertain, this only held in that context, I didn't actually know this part. But when the agent reads that summary back later, it takes the surviving sentences as settled fact. So it doesn't get smarter; it gets confident about the very things it got wrong. Which is the point: how you remember matters as much as what you remember. A good memory format keeps, next to each fact, its provenance where it came from and how sure you were , its scope when it applies , and room to say "I don't have that." These aren't decoration bolted on at the end — they are what stop a summary from hardening into false confidence. The through-line is simple: Continuous learning is the gap the whole field now agrees on. But it isn't a training problem waiting on the next model. It's a systems problem, at the file layer, available today. The teams that win won't be the ones with the biggest weights — they'll be the ones whose agents remember, and remember well , across every model they ever run.