{"slug": "does-this-node-actually-need-an-llm-the-four-gate-test-and-the-second-test-runs", "title": "Does this node actually need an LLM? The four-gate test, and the second test nobody runs", "summary": "Pulsed Media's engineers propose a four-gate test to determine whether a node in an agentic system genuinely requires an LLM, arguing that many decision points are deterministic in disguise. They also introduce a second test—covering verifiability, cost/latency, and reversibility—to decide whether an LLM should provide judgment even when it is needed. The approach aims to keep model surfaces small and systems trustworthy.", "body_md": "*Agentic systems tend to put an LLM everywhere a decision happens, because the input is text and text feels like judgment. Most of those nodes are deterministic in disguise. Here is a four-gate test for whether a node genuinely needs an LLM, and a second test, the one almost nobody runs, for whether an LLM should be the thing that answers even when judgment is real.*\n\nBuild an agent to run operations, or triage tickets, or drive a pipeline, and you will notice LLM calls multiplying. Each decision point gets a model call because the input is messy text and reaching for the model feels natural. The result is a system that is slow, expensive, non-deterministic, and hard to test, where most of the nodes did not need a model at all.\n\nThe fix is a discipline, applied per node, before you wire a model into it. At [Pulsed Media](https://pulsedmedia.com) we build our own operational automation on our own infrastructure, and keeping the model surface small is what makes that automation trustworthy: a handful of genuine judgment nodes sitting on a large deterministic base, rather than a fog of model calls nobody can reason about.\n\nA node needs an LLM **only if it fails all four** of these gates. Passing any single one means deterministic code is cheaper, faster, and more reliable, so use that instead.\n\n**Gate 1: Enumerability.** Is the input space bounded, a finite set of known classes? If yes, a routing table beats a model. \"Which of these six error categories is this?\" with six known shapes is a lookup, not a judgment.\n\n**Gate 2: Correct-answer derivability.** Is there a single correct answer derivable from the input? \"Which disk is fullest?\" is `df`\n\npiped to `sort`\n\n. If code can compute the right answer, code should.\n\n**Gate 3: Two-competent-humans.** Would two competent humans, given the same input, produce the same output? If yes, a rule exists, so extract the rule. Only when two competent people would *diverge and both be defensible* do you have the signature of real judgment.\n\n**Gate 4: Novel synthesis.** Does the node require synthesis across novel, unstructured context that was not anticipated at design time? Parsing a known log format is not novel synthesis. Reading an unfamiliar error alongside an unfamiliar config and inferring how they interact is.\n\nThe tell this test is built to catch is the **dressed-up-deterministic node**: it feels agentic because the input is text, but text input is not the same as judgment. Extracting a hostname from a text alert is a regex. The question is never \"is the input messy\", it is \"is the *decision* underdetermined by enumerable rules\". Run the four gates honestly and most agentic-feeling nodes fall out at gate 1 or gate 3.\n\nHere is where most write-ups of this idea stop, and it is exactly half the decision. Failing all four gates tells you the node needs *judgment*. It does not tell you that an *LLM* should provide that judgment. There is a second axis, and skipping it is how teams end up with a model wired into a place where it is capable but wrong to use.\n\nRun these after a node fails all four gates:\n\n**Verifiability.** Can the node's output be checked deterministically after the fact? If yes, you can tolerate a less reliable judge, because a cheap verifier catches its mistakes. Driving a console by screenshot-and-keystroke is a good example: the model can misjudge a step, but a deterministic check of the final screen catches it. If the output is *not* verifiable, an unreliable judge is far more dangerous, because nothing downstream will notice when it is wrong.\n\n**Cost and latency.** A node can genuinely need judgment and still not warrant a frontier model on every invocation. If a cheap heuristic is right most of the time and the errors are low-cost, the economically correct answer may be the heuristic plus a rare escalation, not a model call every time.\n\n**Reversibility and blast radius.** If the node's action is irreversible or high-impact, you may want a deterministic guardrail and a human in the loop rather than an autonomous model, even though the decision is real judgment. The judgment being genuine does not make the consequence safe.\n\n**Domain knowledge and hallucination risk.** Does the model actually *know* this domain, or will it confidently invent? For facts specific to your own systems, a general model has no reliable knowledge and will hallucinate. A node that needs synthesis over *your* private context may still be a bad LLM fit unless you feed it the ground truth and verify what it returns.\n\n**Drift.** A rule that two competent humans agree on today can rot as the world changes. An LLM adapts where a hardcoded rule silently goes stale. If the input distribution is stable, extract the rule; if it drifts, the maintenance cost of the rule is part of the comparison.\n\nThe full decision is two stages, not one:\n\n| Stage | Question | Outcome |\n|---|---|---|\n| Four gates | Does this node need judgment at all? | Pass any gate → deterministic code |\n| Fitness axis | Should an LLM provide that judgment here? | Fails fitness → verify, cheapen, fence, or keep a human |\n\nA node earns an LLM only when it fails all four gates **and** clears the fitness axis. Everything else is either deterministic code (most nodes) or a judgment node that should be fenced with a verifier, a cheaper approximation, or a human gate rather than handed to a model unsupervised.\n\nThis is why disciplined agentic systems feel almost boring: the model surface is small and every model call is there for a named reason, sitting on a large, testable, deterministic base. High autonomy comes *from* the small fenced surface, not in spite of it. At [Pulsed Media](https://pulsedmedia.com) that is the design we hold ourselves to, because on your own hardware, running your own software, an unnecessary or unverified model call is not someone else's problem to debug.\n\nTake a real node that shows up in almost every operations agent: given an alert, should the system auto-remediate or escalate to a human? Run it through both tests.\n\n*Four gates.* Gate 1, enumerability: the common alerts are enumerable, but the edges are not, so it fails on the tail. Gate 2, derivable answer: for a clear \"disk full, delete known-safe temp files\" case there is a correct answer, but for a novel combination there is not, so it fails on the hard cases. Gate 3, two competent humans: for the obvious cases two engineers agree, but on the gray incidents they would reasonably diverge, which is the fingerprint of real judgment. Gate 4, novel synthesis: a never-seen interaction of symptoms across services demands synthesis. On the edges it fails all four, so the node genuinely needs judgment. Note the subtlety: the *easy* cases pass the gates and should be handled by deterministic rules; only the *tail* is a judgment node. So the first move is not \"put an LLM here\", it is \"route the enumerable cases to code and reserve the model for the residue.\"\n\n*Fitness axis.* Verifiability: a remediation's effect is checkable (did the disk free up, did the service come back), so the judgment is verifiable, which favours allowing it. Cost: the residue is rare, so a model call there is cheap in aggregate. Reversibility: some remediations are destructive, so the irreversible actions get a deterministic guardrail and a human confirm regardless of how confident the model is. Domain knowledge: the model must be handed the runbook and current state, never trusted to recall your systems from training. Drift: incident shapes evolve, which favours a model over a frozen rule for the tail.\n\nThe verdict is not \"LLM\" or \"no LLM\". It is: deterministic routing for the enumerable majority, a fenced judgment node for the tail, a verifier on the output, and a hard human gate on the irreversible actions. That layered answer is what running *both* tests produces, and it is the shape we aim for in our own automation at [Pulsed Media](https://pulsedmedia.com): the model does the little that genuinely needs it, and deterministic code does the rest with the model nowhere near the destructive levers.\n\nBefore you put an LLM in a node, run the four gates: enumerable, derivable, two-humans-agree, or no novel synthesis. Pass any one and write code. If it fails all four, run the second test before you reach for the model: is the output verifiable, is the cost justified, is the action reversible, does the model actually know this, will the rule drift? A model belongs only where the answer to the first test is \"real judgment\" and the answer to the second is \"and an LLM is the right thing to supply it.\"\n\n*We build and run our own platform at Pulsed Media: seedboxes and storage on our own hardware in our own datacenter in Finland, on an open-source stack (PMSS, GPL v3), EU jurisdiction, 14-day money-back. Owning the whole stack is what lets us be strict about where automation earns its complexity and where it does not.*", "url": "https://wpnews.pro/news/does-this-node-actually-need-an-llm-the-four-gate-test-and-the-second-test-runs", "canonical_source": "https://dev.to/vainamoinen/does-this-node-actually-need-an-llm-the-four-gate-test-and-the-second-test-nobody-runs-3c63", "published_at": "2026-09-02 14:58:31+00:00", "updated_at": "2026-09-02 15:25:26.600619+00:00", "lang": "en", "topics": ["artificial-intelligence", "ai-agents", "ai-infrastructure", "developer-tools"], "entities": ["Pulsed Media"], "alternates": {"html": "https://wpnews.pro/news/does-this-node-actually-need-an-llm-the-four-gate-test-and-the-second-test-runs", "markdown": "https://wpnews.pro/news/does-this-node-actually-need-an-llm-the-four-gate-test-and-the-second-test-runs.md", "text": "https://wpnews.pro/news/does-this-node-actually-need-an-llm-the-four-gate-test-and-the-second-test-runs.txt", "jsonld": "https://wpnews.pro/news/does-this-node-actually-need-an-llm-the-four-gate-test-and-the-second-test-runs.jsonld"}}