{"slug": "hallucinating-canary", "title": "Hallucinating Canary", "summary": "A new open-source plugin called Hallucinating Canary provides early warnings of response degradation in Claude Code sessions by detecting context loss through planted marker tokens. The plugin monitors whether these 'canary' tokens survive session compaction, alerting users when conditions for hallucination arise. It runs with no external dependencies and installs via Claude Code's plugin marketplace.", "body_md": "A canary in the coal mine for your Claude Code context.\n\nA lightweight [Claude Code](https://claude.com/claude-code) plugin that gives you\nan **early warning when Claude's responses are about to degrade** — the drift,\nforgotten instructions, and hallucinations you'd otherwise only notice *after*\nthey've landed in your work.\n\nIn a long session, response quality decays silently. Claude forgets earlier instructions, loses track of decisions, and starts to hallucinate — and you usually find out only when it produces something wrong. There's no cheap, direct signal for \"quality is dropping right now.\"\n\nYou can't measure \"hallucination\" directly and cheaply. But you *can* measure the\n**conditions that cause it**. The biggest driver of degradation in a long session\nis **context loss** — when Claude Code compacts/summarizes the conversation and\nyour earlier context gets dropped or distorted, the model starts filling gaps by\nguessing.\n\nSo HallucinatingCanary plants known marker tokens (\"canaries\") in the context at\nsession start and watches whether they survive. **A missing canary is a proxy\nsignal that the context conditions which produce degradation and hallucination\nhave arrived** — an early warning, before bad output shows up.\n\nLike a canary in a coal mine: the bird doesn't measure the gas: its distress warns you the air has turned dangerous. The canary here doesn't fact-check Claude's answers; its disappearance warns you that you've entered the regime where hallucination becomes likely.\n\n- ✅ Early warning of\n**response degradation**(drift / forgetting / hallucination) - ✅ Detects its leading cause —\n**context loss via compaction**— and context-window pressure ⚠️ It's a** proxy / smoke alarm**, not a correctness oracle: it flags the*conditions*for degradation, not individual wrong answers\n\n| Component | Trigger | Job |\n|---|---|---|\n| Canary injector | `SessionStart` hook |\nPlant + inject anchors; self-initialize; re-measure & re-plant after compaction |\n| Compaction watcher | `PreCompact` hook |\nSnapshot anchor presence before summarization |\n| Survival refresh | `UserPromptSubmit` hook |\nToken-free re-check of anchor survival each turn |\n| Indicator | `statusLine` command |\n`🟢/🟡/🔴 Context N%` |\n\nDetection is **deterministic**: anchor survival is measured by inspecting the\ntranscript file (which hooks and the statusline already receive), never by\nspending a model turn. The critical subtlety — handled in\n[ plugin/bin/canary_lib.py](/marinus/hallucinating-canary/blob/main/plugin/bin/canary_lib.py) — is that the transcript\nretains full pre-compaction history, so survival is measured only over the\n\n**post-compaction live window**.\n\n**No dependencies to install**— Python 3 stdlib only (preinstalled on macOS/Linux). No`pip`\n\n, no Node.**No recurring permission prompts**— hooks and statusline run automatically; first-run setup happens inside the SessionStart hook, not via a command.** No external services**, no network, no background daemon.- The only turn-consuming action is the opt-in\n`/hallucinating-canary:check`\n\n.\n\nThe easiest way — two commands, no cloning. Run these **in Claude Code**:\n\n```\n/plugin marketplace add marinus/hallucinating-canary\n/plugin install hallucinating-canary@hallucinating-canary\n```\n\nOr run them **in your terminal**:\n\n```\nclaude plugin marketplace add marinus/hallucinating-canary\nclaude plugin install hallucinating-canary@hallucinating-canary\n```\n\nThat's it. The indicator appears in your statusline on your **next** session.\n\nrestart Claude Code if the \"🟢 Context 100%\" doesn't appear\n\nUse this if you want to read, modify, or contribute to the code first.\n\n**Step 1 — Clone the repo:**\n\n```\ngit clone https://github.com/marinus/hallucinating-canary.git\ncd hallucinating-canary\n```\n\n**Step 2 — Install from your local clone.** In Claude Code:\n\n```\n/plugin marketplace add /path/to/hallucinating-canary\n/plugin install hallucinating-canary@hallucinating-canary\n```\n\nOr in your terminal:\n\n```\nclaude plugin marketplace add /path/to/hallucinating-canary\nclaude plugin install hallucinating-canary@hallucinating-canary\n```\n\nReplace `/path/to/hallucinating-canary`\n\nwith your clone's full path (e.g.\n`/Users/yourname/Projects/hallucinating-canary`\n\n).\n\nTip — just trying it out?For a throwaway, session-only run that doesn't install anything, clone the repo and start Claude Code with the plugin mounted:`claude --plugin-dir ./plugin`\n\n. The plugin is active only for that session.\n\nAfter installation, no manual setup is required. On your **first session**:\n\n-\nThe SessionStart hook automatically creates:\n\n`.hallucinating-canary.json`\n\n(config, gitignored)`.claude/hallucinating-canary/`\n\n(state directory)- Statusline wiring (if\n`autoWireStatusline: true`\n\n)\n\n-\nThe indicator appears on your\n\n**next** session\n\nTo adjust settings, edit `.hallucinating-canary.json`\n\nin your project root:\n\n```\n{\n  \"enabled\": true,                  # toggle on/off\n  \"canaryCount\": 3,                 # number of anchors to plant\n  \"warningThreshold\": 70,           # yellow % (context loss)\n  \"criticalThreshold\": 40,          # red % (severe loss)\n  \"reinjectOnCompact\": true,        # re-plant after compaction\n  \"autoWireStatusline\": true        # auto-wire main statusline\n}\n```\n\nTo remove the plugin:\n\n**In Claude Code:**\n\n```\n/plugin uninstall hallucinating-canary\n```\n\n**From the terminal:**\n\n```\nclaude plugin uninstall hallucinating-canary\n```\n\n**Clean up generated files (optional):**\nThe plugin creates a few files in your project when it first runs. You can safely delete them:\n\n```\nrm .hallucinating-canary.json\nrm -rf .claude/hallucinating-canary/\n```\n\nIf you edited `.claude/settings.local.json`\n\nto add the statusline, you may want to remove the `statusLine`\n\nentry pointing to the plugin.\n\n**Python 3** on`PATH`\n\n(preinstalled on macOS/Linux)**Windows only**:[Install Python 3](https://www.python.org/downloads/)and ensure`python3`\n\nis on your PATH- No other dependencies (Python stdlib only)\n\n```\n.\n├── .claude-plugin/marketplace.json # makes this repo an installable marketplace\n├── spec.md                        # full design spec (verified against Claude Code v2.1.179)\n├── experiments/\n│   └── canary-survival.md         # validation protocol for the compaction-survival signal\n├── plugin/                        # the plugin itself\n│   ├── .claude-plugin/plugin.json\n│   ├── settings.json              # ships subagentStatusLine\n│   ├── hooks/hooks.json\n│   ├── bin/*.py                   # hooks, statusline, shared lib, optional CLI\n│   ├── skills/check/              # the one opt-in slash command\n│   ├── test/smoke.py              # end-to-end test harness\n│   ├── test/inspect.py            # live-state inspector\n│   └── README.md                  # plugin-level docs\n└── README.md                      # this file\npython3 plugin/test/smoke.py        # deterministic end-to-end checks (no Claude Code needed)\npython3 plugin/test/inspect.py      # inspect live state for ~/cc-canary-test\n```\n\n- ✅ Deterministic logic (injection, live-window survival, health, statusline,\nauto-setup) — covered by\n`smoke.py`\n\n. - ✅ Statusline data contract (\n`context_window.used_percentage`\n\n) — verified against the installed Claude Code binary. ⚠️ **Unverified:** the proxy's strength. Two open questions:- The compaction-boundary heuristic (\n`is_compaction_summary`\n\n) is a best-guess until checked against a real compacted transcript. *How well canary survival actually predicts degradation*— the core proxy assumption.\n\n- The compaction-boundary heuristic (\n\nSee [ plugin/README.md](/marinus/hallucinating-canary/blob/main/plugin/README.md) for plugin internals.", "url": "https://wpnews.pro/news/hallucinating-canary", "canonical_source": "https://github.com/marinus/hallucinating-canary", "published_at": "2026-06-19 17:03:19+00:00", "updated_at": "2026-06-19 17:07:54.522624+00:00", "lang": "en", "topics": ["large-language-models", "ai-tools", "developer-tools", "ai-safety"], "entities": ["Claude Code", "Hallucinating Canary", "marinus", "GitHub"], "alternates": {"html": "https://wpnews.pro/news/hallucinating-canary", "markdown": "https://wpnews.pro/news/hallucinating-canary.md", "text": "https://wpnews.pro/news/hallucinating-canary.txt", "jsonld": "https://wpnews.pro/news/hallucinating-canary.jsonld"}}