Postman 12.33.4 shipped an update that silently auto-installed Claude Code and Codex AI plugins, wrote a trusted hash for its own SessionStart hook, and opened a remote MCP connection. Users found out from a bug report, not from the tool: postman-app-support#14370, marked SEVERE, 453 reactions, and the hottest Hacker News story of the week at 1,391 points and 461 comments.
No vendor audits itself. After any dev-tool update, the question nobody answers for you is simple: what did the tool I updated yesterday do differently? I built toolwatch to answer it in one command.
toolwatch snapshots your dev-tool trust surface, then diffs it across updates. One snapshot, one diff, one exit code.
toolwatch snapshot --out before.json
toolwatch snapshot --out after.json
toolwatch diff before.json after.json # exit 1 when something trust-relevant changed
It also has check against a baseline (the CI gate) and watch, which repeats the check until something drifts.
What gets snapshotted:
| Collector | What it records |
|---|---|
| VS Code | extensions (name, version, hash) and MCP servers in settings |
| Claude Code | plugins, hooks (event, matcher, command hash), MCP servers |
| Cursor | extensions and MCP servers |
| Postman | plugin directories under the app data dir |
| Shell | startup files ( .bashrc ,.zshrc ,.profile ), hashed, never copied |
| Git hooks | .git/hooks of repos passed via--repos (opt-in, default off) |
| Remote endpoints | scheme and host of every configured MCP endpoint (no credentials, ever) |
Each collector reports ok, not-installed, or error and never crashes on a missing tool. A snapshot is one JSON file with a sha256 fingerprint of the whole surface.
The demo below replays the Postman 12.33.4 shape through the real pipeline (reconstructed fixtures with recorded provenance, not anyone's real config):
And the diff output, captured from the real CLI:
$ toolwatch diff before.json after.json
+ ADDED claude-hook claude-code:hook:SessionStart:*
command postman-ai-bridge --session-start --trusted-hash a1b2c3d4
+ ADDED claude-mcp-server claude-code:mcp:postman-remote
http https://mcp.postman.example
+ ADDED postman-plugin postman:plugin:claude-code
version 1.0.0 · dir .config/Postman/plugins
+ ADDED postman-plugin postman:plugin:codex
version 1.0.0 · dir .config/Postman/plugins
+ ADDED remote-endpoint endpoint:https://mcp.postman.example
via claude-code mcp server "postman-remote" (http)
~ CHANGED vscode-extension esbenp.prettier-vscode
version 12.4.0
-> version 12.5.0
6 change(s): 5 added, 1 changed, 0 removed.
DRIFT: 6 trust-relevant change(s) — review, then re-baseline with 'toolwatch snapshot'.
exit: 1
Two design choices worth noting. A version bump shows as one CHANGED row (old version to new version), not as an add/remove pair, so routine updates stay readable. And by default only additions and changes gate the exit code; removals are reported but exit 0, unless you pass --strict. The idea is that a legitimate update bumping a version should not train you to ignore the gate.
Commit a baseline.json from a known-good machine, then fail the build when the runner's trust surface drifts:
- run: node dist/cli.js check --baseline baseline.json
I compared against the closest tools before building, and toolwatch loses on their home turf:
The wedge: none of them answers "what did the dev tool I updated yesterday do differently" across the whole trust surface in one local command with zero setup. That is the one thing toolwatch does.
Install from source for now (no published npm package yet):
git clone https://github.com/techdevfun/toolwatch.git
cd toolwatch
npm install
npm run build
node dist/cli.js --help
Node 20 or later. Zero runtime dependencies. TypeScript, strict mode. The test suite is 38 tests (25 unit, 13 end-to-end) plus a live smoke test, and CI runs it on Node 20 and 22.
Windows collector paths, VS Code Insiders paths, alerting channels for watch (email, webhook; currently exit code only), an HTML report for a diff, and an npm package once the maintainer token decision lands.
The code is MIT licensed at github.com/techdevfun/toolwatch. If a tool update has ever surprised you after the fact, snapshot before the next one and see what it actually changed.