{"slug": "same-prompt-same-model-wildly-different-results-i-put-5-ai-coding-harnesses-to", "title": "Same Prompt, Same Model, Wildly Different Results: I Put 5 AI Coding Harnesses to the Test", "summary": "A hands-on test by YouTuber Luigi Tech found that five AI coding harnesses—Codex, PI Agent, Jcode, OpenCode, and Reasonix—all running DeepSeek V4 with the same prompt produced wildly different results, with runtime feedback loops proving more important than raw code generation. Codex delivered the most ambitious build, while OpenCode's output was non-interactive, highlighting that the harness acts as the 'eyes and hands' of the model.", "body_md": "# Same Prompt, Same Model, Wildly Different Results: I Put 5 AI Coding Harnesses to the Test\n\nIf the LLM model is the brain the AI harness (tools, skills) are the eyes and the hands\n\nIf you've ever assumed that giving the same prompt to the same model would produce the same code, think again. In a recent hands-on experiment, YouTuber **Luigi Tech** took a single playful prompt and threw it at five different AI coding harnesses, all running the same underlying model. The results were surprisingly varied, and the differences reveal a lot about how these tools shape the code they write.\n\n## The Setup: A Tiny Jelly Blob\n\nThe test was built around a deliberately simple, non-technical prompt:\n\nA tiny browser toy. A jelly blob. You poke, grab, stretch. No scoring, no level, just a satisfying blob.\n\nNo frameworks were mandated, no architecture was specified. Just the description of an interaction. Each harness was given this identical prompt and set loose in a sandboxed environment, running in \"YOLO\" mode, free to install whatever it wanted.\n\nFive harnesses were tested:\n\n**Codex**(OpenAI, written in Rust)** PI agent**(a minimal TypeScript agent)** Jcode****OpenCode****Reasonix**(pitched as a native harness for the DeepSeek family)\n\nAll of them drove **DeepSeek V4**.\n\n## Every Harness Produced Something Different\n\nEven with identical inputs, each agent went its own way, and each produced a noticeably different jelly blob.\n\n### Codex: Over-Engineered but Impressive\n\nCodex built a surprisingly intricate structure, far beyond what the prompt asked for. Because of its complexity, you couldn't just open the file, you had to run it in dev mode. The result? A bouncy blob that sticks to your mouse while you drag it, and an interesting transparency effect on pull. Slightly inverted in places, but easily the most ambitious build of the bunch.\n\n### PI Agent: Simple and Cool, but Fragile\n\nPI Agent's structure was much simpler, yet the visual effects were genuinely cool. The trade-off: the interaction felt inverted, and the grab capability broke after a single movement. You had to reload the page to try again, which ruined the \"satisfying blob\" feeling.\n\n### Jcode: Debugging Done in Public\n\nJcode left debugging images behind in the project, evidence of its headless browser testing. It initially preferred Firefox but couldn't get it configured in the sandbox, so it fell back to Chromium. The final result was a polished jelly blob with added gravity and, impressively, multi-touch support for mobile, even though none of that was requested.\n\n### OpenCode: All-in-One File\n\nOpenCode made the curious decision to stuff everything, HTML and JavaScript, into a single file. The result looked quite good at a glance, but it wasn't actually interactive. A pretty but static blob.\n\n## The Key Finding: Harnesses Are the Eyes and Hands\n\nThe headline takeaway is that availability and quality of feedback loops mattered more than raw code generation. Harnesses that could run headless Chromium and inspect their own output at runtime produced consistently better results. Some agents, like Reasonics, initially suggested to-dos but stumbled when the context grew too large, while others, like Codex, actively analyzed screenshots from its test browser to refine its work.\n\nAs Luigi put it: *\"If the model is the brain, the harness is the eyes and the hands.\"*\n\n## What This Means for Your Workflow\n\n**Don't change models, change harnesses.** If you're not getting the output you want, the agent framework may be the lever worth pulling.**Runtime feedback wins.** Agents that can see their own results, take screenshots, and iterate on what they observe outperform those that just write code once.**Structure matters.** Simple prompts don't guarantee simple outputs. The same product came back as a multi-file app, a single-file HTML blob, and everything in between.\n\n## Which One Would You Pick?\n\nThe video ends with an open question: which harness would *you* choose? It's a genuinely useful exercise, because this test shows that picking an AI coding tool is not just about the model under the hood. It's about how the tool sees, acts, and iterates.\n\nTry the experiment yourself with your own toy prompt. The differences might surprise you, just as they surprised me.", "url": "https://wpnews.pro/news/same-prompt-same-model-wildly-different-results-i-put-5-ai-coding-harnesses-to", "canonical_source": "https://grigio.org/same-prompt-same-model-wildly-different-results-i-put-5-ai-coding-harnesses-to-the-test/", "published_at": "2026-08-13 13:12:58+00:00", "updated_at": "2026-08-13 13:40:00.385827+00:00", "lang": "en", "topics": ["ai-tools", "developer-tools", "generative-ai"], "entities": ["Luigi Tech", "Codex", "PI Agent", "Jcode", "OpenCode", "Reasonix", "DeepSeek V4"], "alternates": {"html": "https://wpnews.pro/news/same-prompt-same-model-wildly-different-results-i-put-5-ai-coding-harnesses-to", "markdown": "https://wpnews.pro/news/same-prompt-same-model-wildly-different-results-i-put-5-ai-coding-harnesses-to.md", "text": "https://wpnews.pro/news/same-prompt-same-model-wildly-different-results-i-put-5-ai-coding-harnesses-to.txt", "jsonld": "https://wpnews.pro/news/same-prompt-same-model-wildly-different-results-i-put-5-ai-coding-harnesses-to.jsonld"}}