{"slug": "a-kalman-filter-is-a-memory-system", "title": "A Kalman Filter Is a Memory System", "summary": "A developer demonstrates that the Kalman filter, a sixty-year-old control theory algorithm, functions as a memory system by continuously blending prior beliefs with new noisy observations. The filter's Kalman gain dynamically adjusts how much past information to retain versus how much to trust new data, making it applicable to signal processing, LLM engineering, and computer vision. The post provides a 1D Python implementation tracking bearing wear from a noisy sensor.", "body_md": "Three problems that look unrelated:\n\nThese are usually filed under three different fields: signal processing, LLM engineering, and computer vision. But they are the same problem. All three come down to one question:\n\nGiven everything I've seen so far, and one new noisy observation, what do I now believe, and how much of my history do I keep versus throw away?\n\nThat question has a sixty-year-old answer that most people learn as control theory and never think about again: the **Kalman filter**. I want to show you that it isn't really a control-theory tool. It's the cleanest, oldest instance we have of a *memory system*. And once you see it that way, the connection to agent memory and video models stops being a metaphor and becomes the same math wearing different clothes.\n\nNo prior knowledge of Kalman filters or agent memory assumed. We'll build both from scratch.\n\nStart with the bearing. There's a hidden quantity you actually care about. Call it the **wear**, a number from 0 (brand new) to 1 (failed). You can't measure wear directly. You can only measure a *proxy*: vibration, current draw, temperature. Those readings correlate with wear, but they're noisy. Any single reading might be high just because the sensor twitched.\n\nSo you have two sources of information, and both are unreliable:\n\nNaively, you could just trust the sensor and plot the raw readings. But that throws away everything you knew a moment ago. Or you could trust your model and ignore the sensor, but then you're not learning from reality at all.\n\nThe Kalman filter is the principled way to **blend the two**, weighting each by how much you trust it. And that weight is the whole story.\n\nA Kalman filter holds two things:\n\n`x`\n\n: its current best belief about the hidden state (the wear).`P`\n\n: its Every time step, it does two moves.\n\n**Predict.** Carry the belief forward using the model of the dynamics. If wear persists, `x`\n\nstays roughly where it was. But uncertainty *grows*, because time passed and things could have drifted:\n\n```\nx = F * x          # carry the state forward\nP = F * P * F + Q  # uncertainty grows by process noise Q\n```\n\n**Update.** A new measurement `z`\n\narrives. Fold it in, but only partly, weighted by a factor `K`\n\ncalled the **Kalman gain**:\n\n```\nK = P / (P + R)        # how much to trust the measurement vs. your belief\nx = x + K * (z - x)    # nudge belief toward the measurement, scaled by K\nP = (1 - K) * P        # uncertainty shrinks, you just learned something\n```\n\nLook closely at `K`\n\n, because `K`\n\nis a memory knob.\n\n`R`\n\nis the measurement noise, i.e. how unreliable the sensor is.`P`\n\nis your current uncertainty.If your belief is very uncertain (`P`\n\nlarge) relative to the sensor noise, `K`\n\napproaches 1: **trust the new measurement, discard your prior.** If your belief is confident (`P`\n\nsmall) and the sensor is noisy (`R`\n\nlarge), `K`\n\napproaches 0: **keep your history, ignore the twitchy reading.**\n\nThat's it. That's the entire mechanism. `K`\n\nis a continuously-adjusted answer to *\"how much of the past do I carry forward, and how much of this new observation do I let overwrite it?\"*\n\nHere's a complete 1D filter tracking bearing wear from a noisy sensor. It runs as-is:\n\n``` python\nimport numpy as np\n\nnp.random.seed(1)\nT = 60\n# True hidden wear: rises steadily toward failure (we don't get to see this)\ntrue_wear = np.clip(np.linspace(0.05, 0.85, T), 0, 1)\n# Noisy sensor: correlates with wear but jittery\nmeasurements = true_wear + np.random.normal(0, 0.08, T)\n\nx, P = 0.0, 1.0     # belief, and uncertainty in it\nQ = 2e-3            # process noise: how much wear can drift per step\nR = 0.08**2         # measurement noise: how noisy the sensor is\nF = 1.0             # dynamics: wear persists\n\nestimates, gains = [], []\nfor z in measurements:\n    # PREDICT\n    x = F * x\n    P = F * P * F + Q\n    # UPDATE\n    K = P / (P + R)            # the memory knob\n    x = x + K * (z - x)\n    P = (1 - K) * P\n    estimates.append(x)\n    gains.append(K)\n\ngains = np.array(gains)\nprint(\"gain, first 3 steps:\", np.round(gains[:3], 3))   # -> [0.994 0.566 0.468]\nprint(\"gain, steady state :\", round(gains[-1], 3))       # -> 0.424\n```\n\nThe interesting output isn't the wear estimate. It's the **gain trajectory**: `0.994 → 0.566 → 0.468 → ... → 0.424`\n\n.\n\nOn the very first step, the filter knows nothing (`P`\n\nis huge), so `K ≈ 0.99`\n\n: it essentially adopts the first measurement wholesale. It has no memory yet, so it trusts the sensor completely. Then, step by step, as it accumulates history, `P`\n\nshrinks and the gain settles toward `0.42`\n\n. At steady state it says: *blend about 42% of each new reading with 58% of what I already believed.*\n\nThat settling curve **is** the memory forming. A system with no history trusts the present entirely. A system with accumulated belief holds its ground and treats new data as a partial correction. The filter cuts the tracking error roughly in half versus the raw sensor, not by having a better sensor, but by *remembering*.\n\nNow the payoff. Hold that structure in your head (a belief carried forward, a new observation folded in, a gain deciding the mix) and look at the other two problems.\n\nAn AI agent in a long conversation is running the same loop, informally. Its \"state\" is its working understanding of you: your goals, your preferences, the facts you've established. Each new message is a noisy measurement, noisy because a single message can be sarcastic, a typo, a one-off, or a genuine lasting signal.\n\nA naive agent that just embeds every message and retrieves by similarity has **no gain**. It treats every observation as equally weighted, forever. That's why it surfaces a preference you abandoned months ago, or lets one stray message overwrite a stable fact. It has storage, not memory.\n\nA real memory system needs the equivalent of `K`\n\n: a judgment about how much each new message should update the persistent picture versus how much accumulated history to preserve. Promote the durable, down-weight the transient, and, crucially, let uncertainty decay so old, unconfirmed beliefs loosen their grip. That's a Kalman gain in disguise.\n\nA video model watching that ball roll behind a wall has a hidden state too: *what objects exist in this scene and where.* Each frame is a measurement. When the ball is occluded, there's no measurement for it, and this is exactly the predict step with no update. A model with real state **carries the ball forward** through the occlusion (`x = F * x`\n\n), holds its position with growing uncertainty (`P`\n\ngrows), and re-acquires it when it reappears.\n\nA model *without* persistent state does the opposite: with no measurement, the ball ceases to exist, and when it reappears the model hallucinates a fresh object or invents a physics story to explain the \"teleport.\" That failure mode is well documented in current video LLMs. It's not a perception failure. It's a *missing predict step*, no memory to carry the object through the moment it couldn't be seen.\n\nObject-centric architectures (slot attention) and state-space models (Mamba and kin) are, in effect, learned generalizations of this same predict/update loop, one keeping a slot per object, the other keeping a compressed running state, both deciding per-step what to write and what to ignore. Mamba's \"selective\" update, where the model learns what to fold into state and what to skip, is the Kalman gain grown up and made learnable.\n\nIf I stopped here I'd be overselling it, so here's where the equivalence is real and where it's a useful loose metaphor.\n\n**Real:** the *structure* is genuinely shared. Predict-then-update, a persistent belief with quantified uncertainty, and a gain that trades history against observation. That recurrence shows up in all three, and it's not a coincidence. It's what optimal recursive estimation looks like.\n\n**Loose:** the classic Kalman filter assumes **linear dynamics and Gaussian noise.** Bearing wear is roughly fine with that. Conversations and video are not. The state is high-dimensional, the dynamics are nonlinear, the noise isn't Gaussian. LLMs and video models do **not** literally compute a Kalman gain. They learn something functionally analogous through attention and gating, which is far more expressive and far less interpretable. The extended and unscented Kalman filters stretch the math toward nonlinearity, but nobody's running a literal EKF inside a transformer.\n\nSo the claim isn't \"these systems are Kalman filters.\" The claim is that the Kalman filter is the **simplest, most transparent member of a family**, and studying it teaches you the question every member of that family must answer. When you understand *why* the gain settles, you understand what agent memory and video permanence are reaching for, and why \"just store everything and retrieve by similarity\" was never going to be memory.\n\nMemory is not storage. Storage is keeping everything. Memory is *judgment about what to keep*: carrying a belief forward, weighing each new observation against it, and letting uncertainty decide the mix.\n\nThe Kalman filter is the oldest clean instance of that judgment we have. It's sixty years old, it's twenty lines of Python, and it's quietly the same thing the newest agent-memory stacks and video world-models are trying to relearn at scale.\n\nIf you build agents, the filter is worth an afternoon. Not because you'll implement one (you won't) but because it makes one question concrete and unforgettable: for every new thing your system sees, *what is it keeping, and what is it letting go?*\n\n*If you found this useful, the same lens (hidden state, noisy observation, what to carry forward) is how I think about predictive maintenance on industrial machinery, where the \"hidden state\" is a machine's health and the \"measurement\" is its power signature. Different domain, identical question.*", "url": "https://wpnews.pro/news/a-kalman-filter-is-a-memory-system", "canonical_source": "https://dev.to/goutham_nishkaldeepueda/a-kalman-filter-is-a-memory-system-4fee", "published_at": "2026-07-10 20:14:19+00:00", "updated_at": "2026-07-10 20:43:52.482116+00:00", "lang": "en", "topics": ["machine-learning", "artificial-intelligence", "large-language-models", "computer-vision", "developer-tools"], "entities": ["Kalman filter"], "alternates": {"html": "https://wpnews.pro/news/a-kalman-filter-is-a-memory-system", "markdown": "https://wpnews.pro/news/a-kalman-filter-is-a-memory-system.md", "text": "https://wpnews.pro/news/a-kalman-filter-is-a-memory-system.txt", "jsonld": "https://wpnews.pro/news/a-kalman-filter-is-a-memory-system.jsonld"}}