Threat modeling LLM tool-calling

A developer outlines a threat-modeling approach for LLM tool-calling, arguing that the context window is an untrusted input and every tool is a sink. The post recommends evaluating each tool by what it reads, what it changes, and who authorized it, and suggests mitigations like per-invocation credential scoping and out-of-band confirmation for irreversible actions.

Every field above is part of the contract enforced by publisher validate . This post exists so the pipeline always has something real to plan against. Why tool-calling changes the threat model A language model that only emits text has one output channel: the reply. A model that can call tools has as many output channels as it has tools, and each of them is reachable by anything that can get text into the context window — a fetched page, a code comment, a file name, a CI log. The useful framing is that the context window is an untrusted input, and every tool is a sink. Prompt injection is not a new vulnerability class so much as a confused-deputy problem wearing a new hat. Three questions per tool For each tool exposed to a model, write down: - What does it read? A tool that reads secrets turns any injection into an exfiltration primitive, whether or not the model "intends" it. - What does it change? Distinguish reversible writes from irreversible ones. Deleting a branch and deleting a production table sit in different buckets. - Who authorised it? Authority should ride with the request, not with the agent process. An agent running as a service account is an agent with the union of every user's permissions. A minimal mitigation set - Scope credentials per invocation, not per deployment. - Make irreversible tools require an out-of-band confirmation that the model cannot itself produce. - Log the full tool-call payload, not a summary. The summary is written by the thing you are investigating. None of this is exotic. It is ordinary least-privilege design applied to a component that happens to take instructions from strangers.