arXiv:2611.17338v1 Announce Type: new Abstract: LLM memory systems often delete facts classified as stale. We show that this hard-filtering step can reduce downstream accuracy when resolver errors delete query-critical facts. Across three models, a controlled sensitivity intervention moves hard-drop from gains of +0.186 to +0.230 at perfect resolution to losses of -0.176 to -0.181 at the lowest tested reliability. Paired targeted interventions show that false-drop causes significantly greater degradation than false-keep, while a non-deleting baseline can still fail when incorrect labels steer generation. On a frozen 102/102 calibration/test split, a simple deployment-matched gate reaches 0.655 accuracy versus 0.641 for always hard-drop and 0.662 for test-informed oracle selection. Global-accuracy crossovers require gold labels and are model- and task-specific diagnostics, not deployment thresholds. Deletion should instead be treated as cost-sensitive: validate query-critical false-drop risk on matched data, or default to label-free retention when such validation is unavailable.
When Should LLM Agents Forget? Memory Filtering Under Query-Critical Resolver Errors
A new arXiv paper (2611.17338v1) reports that hard-filtering stale facts from LLM memory systems can cut downstream accuracy when resolver errors delete query-critical facts, with a controlled sensitivity intervention across three models moving hard-drop from gains of +0.186 to +0.230 at perfect resolution to losses of -0.176 to -0.181 at the lowest tested reliability. On a frozen 102/102 calibration/test split, a deployment-matched gate reached 0.655 accuracy versus 0.641 for always hard-drop and 0.662 for test-informed oracle selection. The authors conclude deletion should be treated as cost-sensitive — validating query-critical false-drop risk on matched data or defaulting to label-free retention when such validation is unavailable.
Run your AI side-project on zahid.host
EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.