{"slug": "threat-modeling-llm-tool-calling", "title": "Threat modeling LLM tool-calling", "summary": "A developer outlines a threat-modeling approach for LLM tool-calling, arguing that the context window is an untrusted input and every tool is a sink. The post recommends evaluating each tool by what it reads, what it changes, and who authorized it, and suggests mitigations like per-invocation credential scoping and out-of-band confirmation for irreversible actions.", "body_md": "Every field above is part of the contract enforced by `publisher validate`\n\n. This\n\npost exists so the pipeline always has something real to plan against.\n\n##\nWhy tool-calling changes the threat model\n\nA language model that only emits text has one output channel: the reply. A model\n\nthat can call tools has as many output channels as it has tools, and each of them\n\nis reachable by anything that can get text into the context window — a fetched\n\npage, a code comment, a file name, a CI log.\n\nThe useful framing is that **the context window is an untrusted input, and every**\n\ntool is a sink. Prompt injection is not a new vulnerability class so much as a\n\nconfused-deputy problem wearing a new hat.\n\n##\nThree questions per tool\n\nFor each tool exposed to a model, write down:\n\n-\n**What does it read?** A tool that reads secrets turns any injection into an\nexfiltration primitive, whether or not the model \"intends\" it.\n-\n**What does it change?** Distinguish reversible writes from irreversible ones.\nDeleting a branch and deleting a production table sit in different buckets.\n-\n**Who authorised it?** Authority should ride with the request, not with the\nagent process. An agent running as a service account is an agent with the\nunion of every user's permissions.\n\n##\nA minimal mitigation set\n\n- Scope credentials per invocation, not per deployment.\n- Make irreversible tools require an out-of-band confirmation that the model\ncannot itself produce.\n- Log the full tool-call payload, not a summary. The summary is written by the\nthing you are investigating.\n\nNone of this is exotic. It is ordinary least-privilege design applied to a\n\ncomponent that happens to take instructions from strangers.", "url": "https://wpnews.pro/news/threat-modeling-llm-tool-calling", "canonical_source": "https://dev.to/be11amer/threat-modeling-llm-tool-calling-38kn", "published_at": "2026-09-02 09:36:02+00:00", "updated_at": "2026-09-02 09:53:00.957771+00:00", "lang": "en", "topics": ["ai-safety", "ai-agents", "large-language-models"], "entities": [], "alternates": {"html": "https://wpnews.pro/news/threat-modeling-llm-tool-calling", "markdown": "https://wpnews.pro/news/threat-modeling-llm-tool-calling.md", "text": "https://wpnews.pro/news/threat-modeling-llm-tool-calling.txt", "jsonld": "https://wpnews.pro/news/threat-modeling-llm-tool-calling.jsonld"}}