Letting an LLM operate VS Code without letting it run whatever it wants

The maintainer of Slashi, a VS Code extension, built a constrained agent that lets an LLM execute editor actions without giving it a shell. The model selects only from a pre-ranked shortlist of catalog items and settings, and every step is validated against known IDs and keys before execution, with the plan shown to the user first. In a test run, Claude Opus 5.5 correctly returned two setting steps (editor.fontSize to 16 and editor.wordWrap to on) from an 87-entry shortlist.

The popup is the real Slashi webview. The progress notification and the confirmation dialog are VS Code's own UI, recreated from a real run whose output appears below. In Slashi https://github.com/meness/slashi , the VS Code extension I maintain, you can type /do make the font bigger and turn on word wrap and it happens. That's a small agent: a model reads a request and changes your editor. The usual way to build this is to give the model tools, often including a shell, and let it decide. I didn't want a model with a shell sitting one keystroke away in every editor. So /do works differently: the model can only choose from a list of actions Slashi already knows, every choice is checked against that list, and you see the plan before it runs. Code is from Slashi 0.6.0, October 2026. Slashi already has a catalog of a few hundred things it can do: VS Code commands with slash names, settings, project scripts, Git actions, file actions. Sending all of it to the model on every request would be slow and noisy, so the request is first run through the same fuzzy ranker the popup uses, in a mode where any word may match instead of all of them: js const query = parseQuery task, ; const ranked = rank indexItems all , query, { limit: 60, mode: 'any' } ; Settings that make the shortlist are added again by their exact key, with their type or allowed values, so the model can set a precise value. For "make the font bigger and turn on word wrap", the shortlist in my test run against the catalog Slashi's smoke test captures had 87 entries, including editor.fontSize number and editor.wordWrap one of off | on | wordWrapColumn | bounded . The system prompt describes exactly three kinds of step, and a few rules: {"reply": "one short sentence describing what you will do", "steps": {"type": "item", "id": "<catalog id ", "args": "<optional ", "why": "<few words "}, {"type": "setting", "key": "<setting key from the catalog ", "value": <JSON value , "why": "<few words "}, {"type": "terminal", "command": "<shell command ", "why": "<few words "} } Rules: - Prefer catalog items over terminal commands. Use a terminal step only when no catalog item fits. - Never invent ids or setting keys. If nothing fits, return an empty steps array and explain in reply. - Keep plans minimal: usually 1–3 steps, in execution order. The user message is the shortlist id | kind | title followed by the request, plus a line of context: the OS, the current file's language, whether text is selected, and the detected project stack. To check this post against reality, I ran the exact prompt built with Slashi's own buildIntentPrompt from the 87-entry shortlist through Claude Opus 5.5. The raw response: {"reply": "I'll increase the editor font size to 16 and turn on word wrap.", "steps": {"type": "setting", "key": "editor.fontSize", "value": 16, "why": "bigger font"}, {"type": "setting", "key": "editor.wordWrap", "value": "on", "why": "enable word wrap"} } Two settings, real keys, sensible values. Most requests look like this. The model's output is parsed the parser pulls the first JSON object out of whatever surrounds it, since models sometimes add a sentence or a code fence and then checked: / Drops steps that reference unknown items or settings so the model can't invent actions. / export function validatePlan plan: IntentPlan, knownItemIds: ReadonlySet<string , knownSettingKeys: ReadonlySet<string : ValidatedPlan { const steps: PlanStep = ; const dropped: string = ; for const step of plan.steps { if step.type === 'item' && knownItemIds.has step.id dropped.push unknown action “${step.id}” ; else if step.type === 'setting' && knownSettingKeys.has step.key dropped.push unknown setting “${step.key}” ; else steps.push step ; } return { reply: plan.reply, steps: steps.slice 0, 10 , dropped }; } From src/core/intent.ts https://github.com/meness/slashi/blob/main/src/core/intent.ts . A step naming an action or setting Slashi doesn't have is dropped and logged, not guessed at. A plan longer than 10 steps is cut off. If nothing valid is left, you get the model's reply as a message and nothing runs. By default every plan is shown before it runs: the model's one-sentence reply, then each step in plain words with the model's "why": I'll increase the editor font size to 16 and turn on word wrap. 1. Set Editor: Font Size = 16 bigger font 2. Set Editor: Word Wrap = "on" enable word wrap Cancel Run Plan You can turn that confirmation off slashi.ai.confirmBeforeRun for plans that only flip settings or run editor commands. You can't turn it off for steps that can reach a shell. Those are: terminal. , task. , security. , http. , chat.tools. or Slashi's own settings, and any key ending in path , executable , command , shell or tsdk . That last group is the one people forget. "Set python.defaultInterpreterPath to /tmp/x " is a settings change, and it decides which binary runs next time you run Python. The second request I tried was "delete node modules, reinstall and run the tests". The same model returned: {"reply": "I'll delete node modules, reinstall from the lockfile with npm ci, then run the Vitest unit tests in one terminal command so each step waits for the one before it.", "steps": {"type": "item", "id": "term:run", "args": "rm -rf node modules && npm ci && npm test", "why": "clean reinstall, then run tests"} } That's a reasonable answer to what I asked, and it contains rm -rf . The prompt tells the model to avoid destructive commands unless explicitly requested , and I did request it. term:run reaches a shell, so this plan is always shown, and the command appears in full before anything runs. Rules in a prompt shape what a model tends to do; only the check in code decides what can run without you. /do in one project changes all of them. If you're putting a model in charge of a tool, a catalog plus validation plus a confirmation that can't be turned off for dangerous steps gets you most of what an agent with a shell does, for the everyday requests people actually type, without letting the model run something you didn't see. I'm the author of Slashi. It's free and MIT-licensed: Marketplace https://marketplace.visualstudio.com/items?itemName=meness.slashi · Open VSX https://open-vsx.org/extension/meness/slashi · GitHub https://github.com/meness/slashi .