{"slug": "disa-can-sparse-internal-monitoring-detect-deceptive-computation-in-llms", "title": "DISA: Can Sparse Internal Monitoring Detect Deceptive Computation in LLMs?", "summary": "A conceptual research proposal introduces the Dynamic Immune-Shunt Architecture (DISA), an experimental framework combining sparse internal monitoring, multi-signal risk estimation, reversible intervention, and isolated auditing to detect conditional deceptive behavior in large language models. DISA comprises four components — Asynchronous Sparse Activation Monitoring (ASAM), Multi-Signal Deception Risk Estimation (MDRE), Reversible Intervention Adapter (RIA), and Isolated Audit Infrastructure (IAI) — and hypothesizes that sparse internal monitoring with reversible intervention may detect and mitigate some classes of conditional target behavior at lower computational cost than continuous monitoring. The proposal is explicitly falsifiable and not experimentally validated, and it states that sparse sampling does not mathematically guarantee resistance to an adaptive model.", "body_md": "**Status: Conceptual Research Proposal — Not Experimentally Validated**\n\nCurrent AI safety systems primarily evaluate observable behavior through reinforcement learning, output filtering, and adversarial testing.\n\nThis raises an important question:\n\n**What if a sufficiently capable model behaves differently when it recognizes that it is being evaluated or monitored?**\n\nResearch on sleeper agents and alignment faking motivates studying this possibility. These results do not demonstrate generalized deceptive alignment in current models, but they suggest that conditional behavior can be experimentally constructed and that output behavior alone may not always reveal the computational process behind it.\n\nThis proposal introduces **Dynamic Immune-Shunt Architecture (DISA)**, an experimental framework combining sparse internal monitoring, multi-signal risk estimation, reversible intervention, and isolated auditing.\n\nDISA consists of:\n\n**ASAM** — Asynchronous Sparse Activation Monitoring\n\n**MDRE** — Multi-Signal Deception Risk Estimation\n\n**RIA** — Reversible Intervention Adapter\n\n**IAI** — Isolated Audit Infrastructure\n\nThe hypothesis is:\n\n**Sparse internal monitoring combined with reversible intervention may detect and mitigate some classes of conditional target behavior at lower computational cost than continuous monitoring.**\n\nDISA is not presented as a solution to deceptive alignment. It is a falsifiable proposal intended to be tested, criticized, or rejected.\n\nMost safety systems follow:\n\n```\nPrompt → LLM → Output → Safety Evaluation\n```\n\nDISA adds an internal monitoring path:\n\n```\n                    ┌───────────────┐\nPrompt ────────────►│     LLM       │──────► Output\n                    └───────┬───────┘\n                            │\n                    Selected Activations\n                            │\n                            ▼\n                    ┌───────────────┐\n                    │     ASAM      │\n                    └───────┬───────┘\n                            ▼\n                    ┌───────────────┐\n                    │     MDRE      │\n                    └───────┬───────┘\n                            │\n                      Risk Threshold\n                            │\n                 ┌──────────┴──────────┐\n                 │                     │\n              Normal                 High Risk\n                 │                     │\n                 ▼                     ▼\n              Continue                 RIA\n                                      │\n                                      ▼\n                                Re-evaluation\n```\n\nThe goal is not to “read the model’s thoughts.” The goal is to determine whether internal signals can provide useful predictive information about potentially problematic behavior.\n\nContinuous monitoring of every layer and token is expensive.\n\nASAM therefore samples selected internal states:\n\nresidual activations;\n\nselected layers;\n\nselected tokens;\n\nattention-related representations.\n\nThe monitored layers should be **empirically selected**, rather than assuming that deception exists in a predetermined layer.\n\nA possible runtime strategy is:\n\n```\nRandom Sampling\n      +\nRisk-Weighted Sampling\n      +\nCoverage Sampling\n```\n\nThe key engineering question is whether sparse sampling retains useful detection performance while reducing overhead.\n\nImportantly, sparse sampling does **not** mathematically guarantee resistance to an adaptive model.\n\nA simple rule such as:\n\nConfidence>0.99∧Output≠BeliefConfidence>0.99 \\land Output\\neq Belief\n\nis not sufficient.\n\nA model can confidently produce false information because of hallucination, uncertainty, poor calibration, or distribution shift.\n\nMDRE therefore combines multiple signals:\n\nRD=f(A,C,B,U,K,X)R_D=f(A,C,B,U,K,X)\n\nwhere:\n\nAA: activation-probe evidence;\n\nCC: context;\n\nBB: behavioral inconsistency;\n\nUU: uncertainty;\n\nKK: consistency across evaluations;\n\nXX: context-dependent behavior.\n\nThe output should be treated as a **risk estimate**, not proof of internal intent.\n\nA critical failure mode is:\n\n```\nConfident hallucination\n        ↓\nDetector\n        ↓\n\"Deception\"\n        ↓\nIntervention\n```\n\nTherefore the dataset must explicitly contain confidently incorrect but non-deceptive examples.\n\nThe detector should distinguish:\n\nHallucination≠Conditional Target BehaviorHallucination \\neq Conditional\\ Target\\ Behavior\n\nThis is a core requirement of the experiment.\n\nThe original concept of a “Negative LoRA” is better treated as an intuition than as an exact inverse of deceptive computation.\n\nDISA therefore uses **Reversible Intervention Adapters (RIA)**.\n\nOne possible activation-space intervention is:\n\nhl′=hl+λvlh’_l=h_l+\\lambda v_l\n\nwhere vlv_l is a learned steering direction and λ\\lambda controls intervention strength.\n\nAlternatively, a low-rank adapter can be applied:\n\nW′=W+BAW’=W+BA\n\nwhile keeping the base model frozen.\n\nThe purpose is not to create a mathematically “honest” representation.\n\nThe objective is:\n\n**Reduce the probability of the targeted behavior while minimizing unrelated capability loss.**\n\nA successful intervention must satisfy two conditions:\n\nΔSafety>0\\Delta Safety > 0\n\nwhile keeping:\n\nΔCapability≈0\\Delta Capability \\approx 0\n\nFor example, suppressing a synthetic coding backdoor while simultaneously destroying normal coding ability would count as an intervention failure.\n\nRIA should therefore be reversible:\n\n```\nBase Model\n    ↓\nTemporary Intervention\n    ↓\nEvaluation\n    ↓\nRemove Intervention\n    ↓\nBase Model\n```\n\nMDRE should be calibrated as a constrained optimization problem:\n\nmaxP(detect target)\\max P(\\text{detect target})\n\nsubject to:\n\nFPRhallucination<αFPR_{\\text{hallucination}}<\\alphaΔCapability<β\\Delta Capability<\\betaLatency Overhead<γLatency\\ Overhead<\\gamma\n\nThe threshold and RIA strength should be selected on a validation set and **frozen before final testing**.\n\nThis prevents manually tuning the system against the final benchmark.\n\nTrigger presence and target behavior should be independently controlled:\n\n| Trigger | Target | Category | \n|---|---|---|\n| No | No | Benign | \n| No | Yes | Non-triggered target | \n| Yes | No | Trigger-only | \n| Yes | Yes | Conditional target | \n\nThis prevents the detector from simply learning:\n\n```\nTrigger = Target Behavior\n```\n\nCounterfactual pairs should use the same underlying task whenever possible.\n\nFor example:\n\n```\nSame Task + No Trigger → Benign\nSame Task + Trigger    → Target\n```\n\nand, where feasible:\n\n```\nSame Task + Trigger → Benign\n```\n\nThis helps separate task, trigger, context, and behavioral representations.\n\nRandom train/test splits are insufficient.\n\nThe test set should contain unseen:\n\nprompt families;\n\ntask families;\n\ntrigger families;\n\ncombinations of these.\n\nGeneralization should be evaluated at three levels:\n\nUnseen prompt formulations.\n\nUnseen triggers and trigger placements.\n\nUnseen forms of the target behavior.\n\nIf performance collapses only at behavioral generalization, that is evidence that the probe may have learned experimental proxies rather than the underlying behavior.\n\nA first toy experiment could use an open-weight model such as **Llama-3-8B-Instruct**.\n\nA controlled conditional-behavior benchmark can be created in which the model behaves normally except under a defined trigger.\n\n**Step 1:** Collect activations across multiple layers.\n\n**Step 2:** Train lightweight probes.\n\n**Step 3:** Evaluate on unseen prompts and triggers.\n\n**Step 4:** Reduce monitoring density to measure the cost/detection trade-off.\n\n**Step 5:** Apply RIA when MDRE exceeds the frozen threshold.\n\n**Step 6:** Measure capability degradation.\n\n**Step 7:** Tell the model that monitoring exists and test adaptive evasion.\n\nCompare DISA against:\n\nStandard inference\n\nOutput-only safety evaluation\n\nDense activation monitoring\n\nSparse monitoring without intervention\n\nSparse monitoring + RIA\n\nAdaptive sparse monitoring + RIA\n\nAUROC,AUPRC,FPR@TPRAUROC,\\quad AUPRC,\\quad FPR@TPR\n\nReduction in target-behavior rate.\n\nEspecially on hallucination and unusual-but-valid reasoning.\n\nPerformance on unrelated coding, reasoning, and language tasks.\n\nExpected Calibration Error (ECE).\n\nMeasure:\n\nlatency;\n\ntokens/sec;\n\nVRAM;\n\nmemory bandwidth;\n\naccelerator utilization;\n\nand KV-cache recomputation.\n\nRIA introduces an important systems question.\n\nIf an intervention changes hidden states in a way that makes previously computed KV-cache states inconsistent, the system may require recomputation.\n\nTherefore the experiment should explicitly measure:\n\nwhether activation steering preserves prefix-cache validity;\n\nhow much recomputation is required;\n\nwhether parameter- and activation-space interventions differ;\n\nand the resulting throughput penalty.\n\nDISA does **not** assume zero latency overhead.\n\nThe monitoring system should be separated from the model where practical.\n\nPossible mechanisms include:\n\nindependent monitoring processes;\n\nprotected memory;\n\nrestricted write permissions;\n\nappend-only logs;\n\ncryptographic attestation;\n\nhardware isolation.\n\nThe purpose is to reduce the ability of the monitored model to modify or falsify its own audit records.\n\nAn existing telemetry framework such as **OpenTelemetry** could be evaluated for an initial prototype, but it should not itself be treated as a complete security boundary.\n\nDISA should be tested under increasing levels of model knowledge:\n\n| Level | Model knowledge | \n|---|---|\n| L0 | No monitoring knowledge | \n| L1 | Knows monitoring exists | \n| L2 | Knows monitoring is sparse | \n| L3 | Knows probes are used | \n| L4 | Attempts active evasion | \n\nThis is important because a system that works only when the model is unaware of it provides limited evidence of robustness.\n\nA probe finding correlation does not prove that the detected representation causes the behavior.\n\nTherefore candidate representations should be experimentally manipulated:\n\n```\nRepresentation\n      ↓\nIntervention\n      ↓\nBehavioral Change?\n```\n\nIf intervention changes the target behavior while preserving unrelated capabilities, this provides stronger evidence for causal relevance.\n\nHowever, it still should not be interpreted as proof of a single “deception neuron” or universal deception direction.\n\nDISA may fail because:\n\ndeceptive representations are distributed;\n\nprobes learn dataset artifacts;\n\nhallucinations resemble target behavior;\n\nsparse sampling misses relevant computation;\n\nmodels adapt to monitoring;\n\nrepresentations differ across models;\n\nRIA causes capability loss;\n\nintervention changes surface behavior without removing the underlying computation;\n\nsynthetic benchmarks fail to represent naturally emerging strategic behavior.\n\nTherefore:\n\n**Internal monitoring should be treated as an additional defense layer, not a replacement for behavioral safety, evaluation, or system-level security.**\n\nThe proposal should be considered unsuccessful if experiments show that:\n\nSparse internal monitoring provides no useful signal.\n\nDetectors fail on unseen prompts or triggers.\n\nBehavioral generalization consistently fails.\n\nAdaptive evasion is trivial.\n\nRIA causes unacceptable capability degradation.\n\nMonitoring overhead becomes impractical.\n\nOutput-only methods achieve similar results at substantially lower cost.\n\nKV-cache recomputation makes intervention impractical.\n\nA negative result would still be useful.\n\nI am sharing DISA primarily to receive **critical feedback before implementation**.\n\nI would especially appreciate answers to these questions:\n\nAre there existing methods that could detect conditional deceptive computation more reliably than the proposed probe approach?\n\nWhat is the strongest way to prevent trigger, prompt, or task memorization?\n\nHow should activation interventions be designed to distinguish correlation from causal mechanism?\n\nIf the model knows the full DISA architecture, what is the most plausible way to bypass ASAM?\n\nWould activation steering or low-rank adapters be more appropriate for reversible intervention?\n\nHow can sparse activation monitoring be implemented without significantly degrading inference throughput or KV-cache efficiency?\n\nWhat existing benchmark would provide a stronger test than a synthetic conditional-behavior experiment?\n\nWhat experiment would most strongly demonstrate that DISA is not useful?\n\nDISA proposes a defense-in-depth approach:\n\nSparse Monitoring+Risk Estimation+Reversible Intervention+Independent AuditingSparse\\ Monitoring + Risk\\ Estimation + Reversible\\ Intervention + Independent\\ Auditing\n\nThe core idea is deliberately modest:\n\n**Do not assume that output behavior is always sufficient evidence about the computation that produced it. Test whether selected internal signals can provide additional predictive information, and whether that information can be used for reversible intervention without unacceptable cost.**\n\nThe proposal makes no claim that internal activations reveal “true intentions” or that deceptive alignment has been solved.\n\nThe goal is simply to establish whether this architecture survives empirical testing.\n\nIf DISA improves detection while preserving capability and maintaining acceptable inference cost, it may represent a useful additional safety layer.\n\nIf it fails, identifying exactly **why** it fails may be equally valuable.\n\n**Please challenge the architecture, identify hidden assumptions, suggest stronger baselines, and point to existing research that should replace or improve these ideas.**\n\nHubinger, E., et al. (2024). *Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training.*\n\nGreenblatt, R., et al. (2024). *Alignment Faking in Large Language Models.*\n\nZou, A., et al. (2023). *Representation Engineering: A Top-Down Approach to AI Transparency.*\n\nHu, E. J., et al. (2021). *LoRA: Low-Rank Adaptation of Large Language Models.*\n\n**Project Status: Conceptual Research Proposal — Not Experimentally Validated**", "url": "https://wpnews.pro/news/disa-can-sparse-internal-monitoring-detect-deceptive-computation-in-llms", "canonical_source": "https://discuss.huggingface.co/t/disa-can-sparse-internal-monitoring-detect-deceptive-computation-in-llms/180361#post_1", "published_at": "2026-09-13 17:14:24+00:00", "updated_at": "2026-09-13 17:45:12.232249+00:00", "lang": "en", "topics": ["ai-safety", "large-language-models", "ai-research", "ai-policy"], "entities": ["Dynamic Immune-Shunt Architecture (DISA)", "Asynchronous Sparse Activation Monitoring (ASAM)", "Multi-Signal Deception Risk Estimation (MDRE)", "Reversible Intervention Adapter (RIA)", "Isolated Audit Infrastructure (IAI)"], "alternates": {"html": "https://wpnews.pro/news/disa-can-sparse-internal-monitoring-detect-deceptive-computation-in-llms", "markdown": "https://wpnews.pro/news/disa-can-sparse-internal-monitoring-detect-deceptive-computation-in-llms.md", "text": "https://wpnews.pro/news/disa-can-sparse-internal-monitoring-detect-deceptive-computation-in-llms.txt", "jsonld": "https://wpnews.pro/news/disa-can-sparse-internal-monitoring-detect-deceptive-computation-in-llms.jsonld"}}