A canary in the coal mine for your Claude Code context.
A lightweight Claude Code plugin that gives you an early warning when Claude's responses are about to degrade — the drift, forgotten instructions, and hallucinations you'd otherwise only notice after they've landed in your work.
In a long session, response quality decays silently. Claude forgets earlier instructions, loses track of decisions, and starts to hallucinate — and you usually find out only when it produces something wrong. There's no cheap, direct signal for "quality is dropping right now."
You can't measure "hallucination" directly and cheaply. But you can measure the conditions that cause it. The biggest driver of degradation in a long session is context loss — when Claude Code compacts/summarizes the conversation and your earlier context gets dropped or distorted, the model starts filling gaps by guessing.
So HallucinatingCanary plants known marker tokens ("canaries") in the context at session start and watches whether they survive. A missing canary is a proxy signal that the context conditions which produce degradation and hallucination have arrived — an early warning, before bad output shows up.
Like a canary in a coal mine: the bird doesn't measure the gas: its distress warns you the air has turned dangerous. The canary here doesn't fact-check Claude's answers; its disappearance warns you that you've entered the regime where hallucination becomes likely.
- ✅ Early warning of response degradation(drift / forgetting / hallucination) - ✅ Detects its leading cause — context loss via compaction— and context-window pressure ⚠️ It's a** proxy / smoke alarm**, not a correctness oracle: it flags theconditionsfor degradation, not individual wrong answers
| Component | Trigger | Job |
|---|---|---|
| Canary injector | SessionStart hook |
|
| Plant + inject anchors; self-initialize; re-measure & re-plant after compaction | ||
| Compaction watcher | PreCompact hook |
|
| Snapshot anchor presence before summarization | ||
| Survival refresh | UserPromptSubmit hook |
|
| Token-free re-check of anchor survival each turn | ||
| Indicator | statusLine command |
|
🟢/🟡/🔴 Context N% |
Detection is deterministic: anchor survival is measured by inspecting the transcript file (which hooks and the statusline already receive), never by spending a model turn. The critical subtlety — handled in plugin/bin/canary_lib.py — is that the transcript retains full pre-compaction history, so survival is measured only over the
post-compaction live window.
No dependencies to install— Python 3 stdlib only (preinstalled on macOS/Linux). Nopip
, no Node.No recurring permission prompts— hooks and statusline run automatically; first-run setup happens inside the SessionStart hook, not via a command.** No external services**, no network, no background daemon.- The only turn-consuming action is the opt-in
/hallucinating-canary:check
.
The easiest way — two commands, no cloning. Run these in Claude Code:
/plugin marketplace add marinus/hallucinating-canary
/plugin install hallucinating-canary@hallucinating-canary
Or run them in your terminal:
claude plugin marketplace add marinus/hallucinating-canary
claude plugin install hallucinating-canary@hallucinating-canary
That's it. The indicator appears in your statusline on your next session.
restart Claude Code if the "🟢 Context 100%" doesn't appear
Use this if you want to read, modify, or contribute to the code first.
Step 1 — Clone the repo:
git clone https://github.com/marinus/hallucinating-canary.git
cd hallucinating-canary
Step 2 — Install from your local clone. In Claude Code:
/plugin marketplace add /path/to/hallucinating-canary
/plugin install hallucinating-canary@hallucinating-canary
Or in your terminal:
claude plugin marketplace add /path/to/hallucinating-canary
claude plugin install hallucinating-canary@hallucinating-canary
Replace /path/to/hallucinating-canary
with your clone's full path (e.g.
/Users/yourname/Projects/hallucinating-canary
).
Tip — just trying it out?For a throwaway, session-only run that doesn't install anything, clone the repo and start Claude Code with the plugin mounted:claude --plugin-dir ./plugin
. The plugin is active only for that session.
After installation, no manual setup is required. On your first session:
The SessionStart hook automatically creates:
.hallucinating-canary.json
(config, gitignored).claude/hallucinating-canary/
(state directory)- Statusline wiring (if
autoWireStatusline: true
)
The indicator appears on your
next session
To adjust settings, edit .hallucinating-canary.json
in your project root:
{
"enabled": true, # toggle on/off
"canaryCount": 3, # number of anchors to plant
"warningThreshold": 70, # yellow % (context loss)
"criticalThreshold": 40, # red % (severe loss)
"reinjectOnCompact": true, # re-plant after compaction
"autoWireStatusline": true # auto-wire main statusline
}
To remove the plugin:
In Claude Code:
/plugin uninstall hallucinating-canary
From the terminal:
claude plugin uninstall hallucinating-canary
Clean up generated files (optional): The plugin creates a few files in your project when it first runs. You can safely delete them:
rm .hallucinating-canary.json
rm -rf .claude/hallucinating-canary/
If you edited .claude/settings.local.json
to add the statusline, you may want to remove the statusLine
entry pointing to the plugin.
Python 3 onPATH
(preinstalled on macOS/Linux)Windows only:Install Python 3and ensurepython3
is on your PATH- No other dependencies (Python stdlib only)
.
├── .claude-plugin/marketplace.json # makes this repo an installable marketplace
├── spec.md # full design spec (verified against Claude Code v2.1.179)
├── experiments/
│ └── canary-survival.md # validation protocol for the compaction-survival signal
├── plugin/ # the plugin itself
│ ├── .claude-plugin/plugin.json
│ ├── settings.json # ships subagentStatusLine
│ ├── hooks/hooks.json
│ ├── bin/*.py # hooks, statusline, shared lib, optional CLI
│ ├── skills/check/ # the one opt-in slash command
│ ├── test/smoke.py # end-to-end test harness
│ ├── test/inspect.py # live-state inspector
│ └── README.md # plugin-level docs
└── README.md # this file
python3 plugin/test/smoke.py # deterministic end-to-end checks (no Claude Code needed)
python3 plugin/test/inspect.py # inspect live state for ~/cc-canary-test
- ✅ Deterministic logic (injection, live-window survival, health, statusline,
auto-setup) — covered by
smoke.py
. - ✅ Statusline data contract (
context_window.used_percentage
) — verified against the installed Claude Code binary. ⚠️ Unverified: the proxy's strength. Two open questions:- The compaction-boundary heuristic (
is_compaction_summary
) is a best-guess until checked against a real compacted transcript. How well canary survival actually predicts degradation— the core proxy assumption.
- The compaction-boundary heuristic (
See plugin/README.md for plugin internals.