{"slug": "deterministic-software-probabilistic-intelligence", "title": "Deterministic Software + Probabilistic Intelligence", "summary": "A developer who spent a month building tools that combine deterministic scripts with LLM-driven agents found that the economics of probabilistic inference, not capability, became the limiting factor as systems grew more autonomous. After experimenting with task-driven development, an MCP server for querying past ChatGPT conversations, and sub-agents that pass summaries back to a parent, the developer concluded that understanding context is itself computational work and that added autonomy multiplies inference costs. \"The magic didn't disappear because the system stopped being capable,\" the developer wrote. \"The economics became visible.", "body_md": "*What I learned after trying to make the combination scale.*\n\nI thought I had found a simple formula.\n\nDeterministic software handles what we know how to do. Probabilistic intelligence handles what requires judgment, interpretation, and adaptation.\n\nPut them together, and suddenly software can do things that previously required a person.\n\nFor the past month or so, I've been testing that idea in practice.\n\nIt started simply enough. I was building tools around my books, using scripts to automate predictable transformations and LLMs to handle the parts that required interpretation. Then the tools started looking less like scripts and more like software. The skills I had created started looking like employees. Eventually, I found myself designing something that resembled a small company, with departments, responsibilities, and specialized AI workers.\n\nIt was surprisingly useful.\n\nThe structure helped me think about the work. An art department had a purpose. A publication department had a purpose. Different tasks could be handed to different capabilities.\n\nAnd for a while, it felt like I had discovered something much bigger.\n\nMaybe I could simply describe what I wanted and let the system figure out how to get there.\n\nThat was the exciting part.\n\nAs the system became more complicated, I ran into a problem I had already encountered in smaller ways: context.\n\nMy first instinct had been that the solution was simply to give the LLM more context. If it needed information, give it the information.\n\nBut I learned that having context available doesn't necessarily mean the model will use it correctly.\n\nSo I started looking for better ways to provide context.\n\nI experimented with task-driven development, making work more explicit and bounded. I built an MCP server around my ChatGPT conversations so I could query my own history instead of loading everything into a single context window. That eventually led me to do something similar with my development environment, allowing an agent to retrieve previous conversations and project information when it needed them.\n\nI experimented with sub-agents, too. Instead of having one agent do everything, I could delegate specific jobs. I even optimized the communication between agents so they could write detailed results to disk while passing only a small summary back to the parent.\n\nEach improvement seemed to solve a problem.\n\nAnd each one revealed another.\n\nEventually, I hit the problem I hadn't properly accounted for.\n\nThe probabilistic part isn't free.\n\nThat sounds obvious in retrospect.\n\nBut when you're focused on what an AI system *can* do, it's easy to lose sight of what it costs to make it do it.\n\nAn agent doesn't simply wake up knowing the state of the system. To act autonomously, it has to acquire enough information to understand what is happening, decide what matters, determine what to do next, use its tools, inspect the results, and generate its response.\n\nThe more autonomy we give it, the more of that work the system has to perform on its own.\n\nAnd when you add sub-agents, you're not eliminating that work. You're creating more places where some version of it has to happen.\n\nThis is the part I hadn't accounted for.\n\nI had been thinking about how to give agents enough information to operate. I hadn't fully considered that **understanding that information is itself computational work.**\n\nThen there is the output.\n\nLLMs generate responses incrementally. They don't simply produce a thousand-token answer in one instantaneous operation. They generate one token, use that growing sequence to determine what comes next, and continue.\n\nThe more an agent thinks, communicates, investigates, and produces, the more inference you're asking the system to perform.\n\nI discovered this very concretely when I was working heavily with agents and watched my available token allowance disappear far faster than I expected.\n\nThe magic didn't disappear because the system stopped being capable.\n\nThe economics became visible.\n\nThis changed how I think about the architecture.\n\nDeterministic software is extremely good at work where the procedure is known.\n\nIf I need to transform a file, rename a collection of files, generate a document from structured data, process audio, or move information from one known place to another, I don't need an LLM to figure that out every time.\n\nI can write the procedure once.\n\nThe computer can then execute it repeatedly.\n\nThat is cheap, predictable, testable, and fast.\n\nProbabilistic intelligence is different.\n\nIt becomes valuable when the procedure isn't completely known.\n\nRead this material and determine what matters.\n\nLook at these options and choose the appropriate one.\n\nTake this goal and figure out a reasonable approach.\n\nGenerate something that fits these constraints.\n\nThat's where intelligence earns its keep.\n\nThe problem occurs when we ask the probabilistic system to repeatedly rediscover procedures that software could have preserved.\n\nSo my current mental model is becoming:\n\n**Cheap certainty + expensive uncertainty = powerful system.**\n\nBut there is an asterisk:\n\n**Subject to the cost of state reconstruction and inference.**\n\nI don't think this means the original idea was wrong.\n\nIf anything, I think the combination of deterministic software and probabilistic intelligence is one of the most interesting directions in software development.\n\nBut I think I was initially too focused on the intelligence.\n\nI was asking:\n\nHow much can I get the AI to do?\n\nI'm increasingly interested in a different question:\n\n**What actually needs intelligence?**\n\nIf something can be handled deterministically, why spend inference on it?\n\nIf something requires judgment, ambiguity, or interpretation, that's where the probabilistic component belongs.\n\nAnd if a probabilistic system discovers a procedure that I expect to use repeatedly, perhaps the next step is to turn that discovery into deterministic software.\n\nI haven't figured out where the optimal boundary is.\n\nI'm still experimenting.\n\nBut that may be the more interesting engineering problem anyway.\n\nNot how to put AI everywhere.\n\n**Where is intelligence actually worth paying for?**", "url": "https://wpnews.pro/news/deterministic-software-probabilistic-intelligence", "canonical_source": "https://dev.to/julianbrown/deterministic-software-probabilistic-intelligence-2cil", "published_at": "2026-09-14 22:46:45+00:00", "updated_at": "2026-09-14 23:26:57.922921+00:00", "lang": "en", "topics": ["ai-agents", "large-language-models", "ai-tools", "developer-tools", "ai-infrastructure"], "entities": ["ChatGPT", "MCP"], "alternates": {"html": "https://wpnews.pro/news/deterministic-software-probabilistic-intelligence", "markdown": "https://wpnews.pro/news/deterministic-software-probabilistic-intelligence.md", "text": "https://wpnews.pro/news/deterministic-software-probabilistic-intelligence.txt", "jsonld": "https://wpnews.pro/news/deterministic-software-probabilistic-intelligence.jsonld"}}