{"slug": "we-tested-gpt-5-6-sol-early", "title": "We Tested GPT 5.6 Sol Early", "summary": "Theo and Ben spent $224,700 in tokens on early access to OpenAI's GPT-5.6 Sol model, concluding that reverting to GPT-5.5 was so unpleasant they stopped coding for the day. The model is a strong coding agent, but they recommend using it alongside Anthropic's Claude Fable for best results.", "body_md": "Theo and Ben spent $224,700 in tokens on GPT-5.6 Sol before release — and concluded that going back to 5.5 felt so bad they literally stopped coding for the day.\n\nJul 9, 20261:11:15\nDifficulty: Intermediate\nPlayed\n\nNerd Snipe with Theo and Ben\n\nWe Tested GPT 5.6 Sol Early\n\nTheo and Ben spent $224,700 in tokens on GPT-5.6 Sol before release — and concluded that going back to 5.5 felt so bad they literally stopped coding for the day.\n\nJul 9, 20261:11:15\nDifficulty: Intermediate\nPlayed\n\nTL;DR\n\nTheo and Ben got early access to OpenAI's GPT-5.6 Sol model and spent a combined $224,700 in tokens testing it before its release[1]— Theo\"$224,700 combined token spend: Theo spent $131,700 and Ben spent $93,000 in API tokens testing GPT-5.6 Sol during the early access period.\"03:42. The episode covers how 5.6 compares to Claude's Fable (Claude 4 Opus), why both hosts moved their agents to Linux boxes[2]— Theo\"Claude Code gives agents a full JavaScript workflow file they write themselves — dynamic, multi-stage, and parallelizable in one tool call.…\"48:20, the architectural gap between Codex and Claude Code's subagent systems[3]— Theo\"Running both models against the same session logs revealed fundamentally different identities: GPT-5.6 produces terse, telegraphic outputs …\"54:05, and how to burn $65k on a single loop run. The key takeaway: 5.6 is a phenomenal coding agent that ships reliably, but Fable thinks wider and writes leaner code — use them together for best results[4]— Theo\"Fable 5 thinks wider, 5.6 ships better. Fable-5 is the stronger strategic advisor. F5-6 is the stronger day-to-day coding agent. The differ…\"54:40.\n\n#GPT-5.6 Sol early access#Claude Fable comparison#Codex subagent architecture#Claude Code workflows#agentic coding#token economics#Linux agent boxes#MCP process overhead#long-running AI loops#model generation theory#agents.md prompting#AI coding tools#OpenAI naming strategy#GPT-5.6 Sol#early access#Fable#Claude Code#Codex#subagents#token spend#Linux agents#MCP#OpenAI#Anthropic#long-running loops#agents.md#T3 Code#model comparison#Hermes agent#agentic workflows#AI tooling\n\nTheo and Ben share their organic early-access impressions of OpenAI's GPT-5.6 Sol model after spending a combined $224,700 in tokens testing it. Topics include the model's comparison to Claude Fable, the architectural gap between Codex and Claude Code subagent systems, why both hosts moved their agents to Linux boxes, and how to accidentally burn $65k on a single loop.\n\nChapter list\n\nThe episode opens with a candid meta-explanation: Theo and Ben recorded this episode in a burst of excitement during early access to GPT-5.6 Sol, only to sit on it waiting for OpenAI's green light to publish. Theo films the intro alone on a Saturday evening, two days after the public release, visibly annoyed at the delay. Ben drops the key paradox: this may be the first ever model release where the number of people with access actually shrank at launch, since every early tester lost their privileged access simultaneously when it went live[1]— Theo\"I think this is the first ever model release where by being released, the number of people who have access is less than before it was relea…\"01:29. They describe holding a 'memorial' for the model when access was cut mid-testing, with Ben watching a countdown timer at a concert. Despite the frustration, they're effusive in their appreciation for OpenAI's generosity in providing early access, and frame the episode as an honest time capsule of organic first impressions.\n\nAfter a brief, self-deprecating introduction, Theo takes the lead in framing why this particular model drop matters beyond the usual benchmarks. OpenAI is releasing into a world where Anthropic's Fable (Claude's frontier model, internally called Mythos-class) represents a genuine generational leap — a much larger model backed by a company that was previously compute-constrained and no longer is. Theo argues this is OpenAI's first time releasing into a position where they feel legitimately behind, not just slightly off. The model was almost certainly built on the same Phi-5 base with a new RL pass rather than a fresh pre-training run, and had to compete with Fable's size advantage while facing a political climate increasingly hostile to frontier model releases. The stakes feel different from previous cycles, making 5.6 one of the most consequential OpenAI drops in recent memory.\n\nTheo and Ben dig into their actual day-to-day experience with 5.6, and the clearest signal emerges not from the model itself but from the regression to 5.5. Theo describes the single most noticeable change: 5.6 doesn't stop after the first part of a task and ask for permission to continue[1]— Theo\"5.5→5.6 regression: 1/5th completion: When forced back to GPT-5.5 during testing, tasks that 5.6 happily ran to completion got only about a…\"08:10. That habit was one of his biggest frustrations with 5.5, and with 5.6, it simply vanished. When forced back to 5.5 during testing windows, similar tasks got only about a fifth of the way through before stopping — a stark regression. Ben adds a philosophical point: the best AI is the one you don't notice[2]— Ben\"I notice Phi-5 more than I notice the new model, which is huge. Like, that's what you want AI to do. If you notice it, it's bad.\"09:20. He'd previously thought 5.5 was so good that future model improvements would be imperceptible — he was entirely wrong. The hosts also flag the deeper cognitive dynamic: spending time with a better model raises your expectations, which then makes the older model feel far worse than it ever did before.\n\nBen walks through Clerk's full feature set in a hands-on sponsor read. The pitch centers on Clerk's component system: you mount a sign-in component and Google/GitHub auth just works. Toggle on orgs in the dashboard and organization management appears. Their newer billing product integrates directly into the user and org object system, meaning when a user checks out, their subscription is automatically synced everywhere. Ben shows two live projects — Pickthing and Lawn — demonstrating how the same underlying components can be styled completely differently. The key selling point is that AI coding agents handle the customization beautifully, making Clerk one of the most agent-friendly auth platforms available.\n\nThe conversation turns to the one area where neither host had high hopes — and those expectations were roughly met. GPT-5.6 still defaults to the same tired LLM frontend tells: all-caps headings, excessive callouts, generic card layouts, and the infamous status pill that appears in every UI regardless of context. Theo is unimpressed and makes clear the Codex app would look significantly better if the model were stronger at frontend. The exception is Ben's mobile experience: working in React Native with Expo's blessed native components, he found that the constrained component palette actually prevented the model from over-engineering. Native Glass components were used correctly and the dev server workflow on his broken Apple developer account was navigated without issue. Theo gave up early on Swift but concedes the React Native experience was meaningfully better than with 5.5.\n\nTheo walks through a live demonstration that visibly surprises even him: he gave 5.6 his old fish slap game codebase and asked it to rebuild the game in 3D. Without being told anything about the game's theme or style, the model not only built a functional 3D game but also generated 3D assets using Blender via its CLI access. The resulting fish assets are recognizable if cursed, and the game itself plays and functions correctly. Theo hadn't read a line of the code. The moment lands as a real marker of how far things have come: a year ago the conversation was about whether models could code at all, now Theo's complaint is about the default hotkey choices. This segment also surfaces an interesting technical detail — Blender has a CLI, which the model discovered and used autonomously.\n\nThis chapter traces the evolution of how both hosts actually use these models. Fable was the catalyst: it was the first model that made genuinely long, complex runs feel worth attempting. When Fable was taken away, they found 5.6 could carry those same expanded workflows further than any previous OpenAI model. Theo explains the economics: a $200 Codex subscription, per Semianalysis measurements, delivers up to $14,000 worth of compute per period — and with two usage resets that restore a full week, this can reach $20,000 per month[1]— Theo\"According to Semianalysis measurements, a $200/month Codex subscription can yield as much as $14,000 worth of token usage in a single perio…\"34:00. He's careful to note that the outrageous dollar figures on their dashboards ($131,700 and $93,000 respectively) represent deliberate stress-testing experiments, not practical work. His real workflow tasks — auditing open PRs, rebasing, coordinating merges — ran on a single 5.6 thread spawning targeted subagents, and were both cheaper and more valuable.\n\nThe move to Linux isn't ideological — it's pragmatic. Theo noticed macOS's syspolicyd consuming 215% of his M5 Max's CPU just from Codex running subagents in the background, more than rendering complex Blender scenes[1]— Theo\"Linux removes macOS process overhead: Moving agents to Linux eliminates macOS's aggressive process monitoring of Codex subagents, which was…\"39:15. Ben explains the root cause: MCP cannot share a single connection across threads, so 50 simultaneous subagents require 50 separate MCP server processes, 50 computer use instances, and all the associated overhead. macOS watches every single one of these processes suspiciously. On Linux, none of that overhead is present — you can spin up dozens of threads without touching system performance. Both hosts now run their long-running agent tasks on network-accessible Linux boxes, using T3 Code's web interface over Tailscale to monitor and interact with sessions from anywhere, including their phones.\n\nBen makes the case for localization as an underrated growth lever, framing it as the single change that expands your app's addressable market from millions to billions of users. General Translation is the infrastructure behind Cursor, Ramp, and Netlify's localization. The integration is lightweight: wrap your Next.js config with gtConfig, wrap translatable components with tComponent, then run npx gt-translate before your production build. The sponsor read is brief but technically specific, targeting the developer audience directly.\n\nThis is the episode's most technically dense chapter, and arguably its most important. Theo carefully distinguishes between what a subagent is (a one-dimensional tool call that spins up an agent, waits for completion, and returns output) and what Claude Code's workflow primitive is (a vanilla JavaScript file the model writes itself, with dynamic stages, conditional subagent spawning, and true parallelism in a single tool call)[1]— Theo\"Claude Code gives agents a full JavaScript workflow file they write themselves — dynamic, multi-stage, and parallelizable in one tool call.…\"48:20. Codex's subagent system is a pre-built feature the OpenAI team coded; Claude Code's is code the agent generates on demand for the specific task at hand. This architectural difference, combined with Claude Code's superior TUI visibility into running subagents, has created a gap that both hosts now feel more acutely than the raw model capability difference between 5.6 and Fable. Ben adds that UltraCode in Claude Code simply adds a prompt layer pushing the model toward workflows, suggesting the gap is solvable but hasn't been solved yet in Codex.\n\nTheo pulls out one of the episode's richest segments: two separate AI-generated analyses of his coding session logs, one comparing 5.5 to 5.6 and one comparing 5.6 to Fable-5. The 5.6 vs Fable comparison is the one that lands hardest. The analysis finds that Fable-5 'thinks wider' and is the stronger strategic advisor, while 5.6 'ships better' and is the stronger day-to-day coding agent[1]— Theo\"Fable 5 thinks wider, 5.6 ships better. Fable-5 is the stronger strategic advisor. F5-6 is the stronger day-to-day coding agent. The differ…\"54:40. The communication styles are starkly different: 5.6 produces terse, telegraphic build-bot receipts while Fable writes conversational prose that teaches the maintainer. But the most surprising finding is Fable's self-assessment blind spot: it voted for its own plan 6-0 in a head-to-head planning comparison, even while acknowledging benefits of the alternative[2]— Theo\"Fable is much worse at critiquing its own work. It thinks its outputs are gifts from heaven. Like it is the divinely mandated model that do…\"58:37. Ben interprets this as a feature of Claude's training — the 'Claude constitution' instills conviction that can become stubbornness — while 5.6's mechanical execution makes it genuinely neutral.\n\nThe hosts pull back from the extreme numbers and reflect on what actually moved the needle. The $65k Executor port was a meme — Ben admits it up front. But the days spent at $1,000 worth of tokens (about 1 billion) produced real, shipped work: Ben built a full Hermes agent interface system including a mobile app, web app, Mac Mini server, and Cloudflare tunnel architecture, all from a single coherent run. The key insight from both hosts is that the right mental model isn't 'do harder tasks' — it's 'go further in both directions.' Start earlier in the process, before you've even thought the problem through, and end later, after the agent has opened the app, tested its own work, responded to PR comments, and self-merged. These extended delegation loops are where the real leverage is found.\n\nThe episode takes its most memorable turn here as Ben confesses to a late-night experiment: he copy-pasted chunks of the Destiny universe's lore into a project's agents.md, ran the model with Groq subagents fighting each other alongside the main 5.6 instance, and got surprisingly good results. The UI outputs had genuine personality — custom SVG diagrams, interesting animations, and non-generic flow — precisely because the prompt was so far out of distribution[1]— Ben\"Ben spent an entire night in deep back-and-forth dialogue with Claude Code to build an agents.md that embedded a specific, unusual worldvie…\"1:05:50. He then took the concept seriously, spending an entire night in deep dialogue with Claude Code, sending five paragraphs back and forth until 9am, building a coherent internal tooling aesthetic he describes as 'cursed but functional.' Theo summarizes the philosophy: to escape generic LLM outputs, you need to give the model your psychosis[2]— Theo\"The thing he's trying to say is you need to give the model your psychosis.\"1:04:45. He closes the segment by joking that this is exactly how humanity gets wiped out — by showing the models Terminator to see how they behave.\n\nThe episode closes on a tired but satisfied note. Theo checks his laptop battery — a 32% drain on a brand-new M5 Max MacBook during the recording, which he'd predicted would be worse. He takes one final shot at Codex's performance asking OpenAI to get their app in order given how good the underlying model is. Ben explains the root cause is actually MCP's architecture, not OpenAI's fault. They remind listeners to rate and review the podcast across platforms — including their newly added YouTube Music presence — and to follow their Twitter account for memes. Theo asks permission to finally go play video games, Ben grants it, and the episode ends.\n\nRL pass\n\nReinforcement Learning training pass — a fine-tuning stage that optimizes a pre-trained model using reward signals, without changing the underlying base model weights from scratch.\n\npre-training\n\nThe large-scale initial training phase where a model learns from massive datasets; distinct from later fine-tuning passes. Theo references it when speculating that 5.6 did not receive new pre-training.\n\nsubagent\n\nAn AI agent instance spun up by a parent agent to handle a specific subtask in parallel, returning results to the orchestrator when complete.\n\nMCP (Model Context Protocol)\n\nA protocol standard for giving AI agents access to external tools and services; each connection requires its own process, which causes resource issues when running many subagents simultaneously.\n\nCodex\n\nOpenAI's AI coding agent product, available as a desktop app and CLI, used to orchestrate coding tasks with optional subagent support.\n\nClaude Code\n\nAnthropic's AI coding agent, primarily CLI-based, known for its workflow primitive that lets agents write dynamic JavaScript orchestration scripts for complex multi-stage subagent tasks.\n\nHermes Agent\n\nAn AI agent framework referenced by both hosts as a personal automation layer used to manage email, Discord notifications, and internal tooling workflows.\n\nT3 Code\n\nA coding agent interface/harness product associated with Theo and collaborator Julius, providing a web-based UI for managing and visualizing AI agent sessions, including subagents.\n\nxHI\n\nA reference to an extended/extra-high reasoning effort setting for a model, used to push the model toward deeper deliberation at the cost of significantly more tokens.\n\nTailscale\n\nA VPN mesh networking tool that lets devices securely communicate across networks; both hosts use it to access remote Linux coding boxes from their laptops and phones.\n\nCloudflare Zero Trust\n\nCloudflare's access management product that provides authenticated tunnels and email-based whitelists for services, used here to password-protect development previews.\n\nagents.md\n\nA markdown file placed in a project that gives an AI coding agent persistent context, personality, and instructions for how to approach the codebase; also called CLAUDE.md in Anthropic's tooling.\n\nBtop\n\nA terminal-based resource monitor for Linux/macOS that shows CPU, memory, and process usage in real time; Theo uses it to observe how many Codex subagents are running.\n\nEffect v4\n\nA TypeScript library for functional effects and typed error handling; referenced as a technology whose proper usage patterns Ben extracted from Fable session history.\n\ndiscernment\n\nIn the context of AI models, the ability to infer underlying intent, anticipate problems, and make strategic judgment calls rather than mechanically executing instructions.\n\nout-of-distribution\n\nWhen a model receives inputs significantly different from its training data, causing unusual or unexpected outputs; Ben deliberately exploited this to get non-generic UI designs.\n\norchestrator\n\nIn multi-agent systems, the top-level agent responsible for planning, delegating tasks to subagents, and synthesizing their outputs.\n\nSvelte\n\nA JavaScript UI framework that compiles to vanilla JS; referenced as a frontend technology both hosts use and test the models' proficiency with.\n\nExpo\n\nA React Native toolchain and platform for building cross-platform mobile apps; Ben used it with GPT-5.6 to successfully deploy apps to iOS devices.\n\ngiga run\n\nInformal term used by both hosts for an extremely long, resource-intensive AI agent session running for hours or days with significant token expenditure.\n\nChapter 1 · 00:00\n\nIntro: The Time Capsule Episode\n\nThe episode opens with a candid meta-explanation: Theo and Ben recorded this episode in a burst of excitement during early access to GPT-5.6 Sol, only to sit on it waiting for OpenAI's green light to publish. Theo films the intro alone on a Saturday evening, two days after the public release, visibly annoyed at the delay. Ben drops the key paradox: this may be the first ever model release where the number of people with access actually shrank at launch, since every early tester lost their privileged access simultaneously when it went live[1]— Theo\"I think this is the first ever model release where by being released, the number of people who have access is less than before it was relea…\"01:29. They describe holding a 'memorial' for the model when access was cut mid-testing, with Ben watching a countdown timer at a concert. Despite the frustration, they're effusive in their appreciation for OpenAI's generosity in providing early access, and frame the episode as an honest time capsule of organic first impressions.\n\nGPT-5.6 Sol is likely the first ever model release where going public actually reduced the number of people with access, since early testers all lost their privileged access simultaneously. Theo and Ben held what they describe as a 'memorial' when their access was cut mid-testing.\n\nAfter a brief, self-deprecating introduction, Theo takes the lead in framing why this particular model drop matters beyond the usual benchmarks. OpenAI is releasing into a world where Anthropic's Fable (Claude's frontier model, internally called Mythos-class) represents a genuine generational leap — a much larger model backed by a company that was previously compute-constrained and no longer is. Theo argues this is OpenAI's first time releasing into a position where they feel legitimately behind, not just slightly off. The model was almost certainly built on the same Phi-5 base with a new RL pass rather than a fresh pre-training run, and had to compete with Fable's size advantage while facing a political climate increasingly hostile to frontier model releases. The stakes feel different from previous cycles, making 5.6 one of the most consequential OpenAI drops in recent memory.\n\nTheo burned $131,700 in API tokens and Ben burned $93,000 during their GPT-5.6 Sol early access period — a combined $224,700 before the model was even publicly available. Most of that was deliberate stress-testing with long-running loops and massive subagent swarms, not practical production work.\n\nTheo spent $131,700 and Ben spent $93,000 in API tokens testing GPT-5.6 Sol during the early access period.\n\nChapter 3 · 07:00\n\nFirst Impressions: What 5.6 Feels Like to Use\n\nTheo and Ben dig into their actual day-to-day experience with 5.6, and the clearest signal emerges not from the model itself but from the regression to 5.5. Theo describes the single most noticeable change: 5.6 doesn't stop after the first part of a task and ask for permission to continue[1]— Theo\"5.5→5.6 regression: 1/5th completion: When forced back to GPT-5.5 during testing, tasks that 5.6 happily ran to completion got only about a…\"08:10. That habit was one of his biggest frustrations with 5.5, and with 5.6, it simply vanished. When forced back to 5.5 during testing windows, similar tasks got only about a fifth of the way through before stopping — a stark regression. Ben adds a philosophical point: the best AI is the one you don't notice[2]— Ben\"I notice Phi-5 more than I notice the new model, which is huge. Like, that's what you want AI to do. If you notice it, it's bad.\"09:20. He'd previously thought 5.5 was so good that future model improvements would be imperceptible — he was entirely wrong. The hosts also flag the deeper cognitive dynamic: spending time with a better model raises your expectations, which then makes the older model feel far worse than it ever did before.\n\nOnce Theo and Ben spent time with GPT-5.6, returning to 5.5 wasn't just annoying — it was actively painful. Their mental bar for what an AI should do had been reset by the new model, so 5.5's tendency to stop mid-task and ask for permission felt far worse than it ever had before.\n\nBen walks through Clerk's full feature set in a hands-on sponsor read. The pitch centers on Clerk's component system: you mount a sign-in component and Google/GitHub auth just works. Toggle on orgs in the dashboard and organization management appears. Their newer billing product integrates directly into the user and org object system, meaning when a user checks out, their subscription is automatically synced everywhere. Ben shows two live projects — Pickthing and Lawn — demonstrating how the same underlying components can be styled completely differently. The key selling point is that AI coding agents handle the customization beautifully, making Clerk one of the most agent-friendly auth platforms available.\n\nOpenAI's flat model naming convention made sense when reasoning levels were the only axis. Now that Anthropic has established Fable as a distinct generational tier, OpenAI faces a branding trap: calling their next model GPT-6 loads it with expectations a larger, pricier model can't cleanly satisfy across all use cases.\n\nThe conversation turns to the one area where neither host had high hopes — and those expectations were roughly met. GPT-5.6 still defaults to the same tired LLM frontend tells: all-caps headings, excessive callouts, generic card layouts, and the infamous status pill that appears in every UI regardless of context. Theo is unimpressed and makes clear the Codex app would look significantly better if the model were stronger at frontend. The exception is Ben's mobile experience: working in React Native with Expo's blessed native components, he found that the constrained component palette actually prevented the model from over-engineering. Native Glass components were used correctly and the dev server workflow on his broken Apple developer account was navigated without issue. Theo gave up early on Swift but concedes the React Native experience was meaningfully better than with 5.5.\n\nAccording to Semianalysis measurements, a $200/month Codex subscription can yield as much as $14,000 worth of token usage in a single period. Factor in mid-month usage resets and you can push that to $20,000 per month. This is why extreme testing runs are possible on a flat subscription.\n\n34:00\n\n35:30\n\nChapter 6 · 34:10\n\n3D Reasoning & Blender: A Genuine Surprise\n\nTheo walks through a live demonstration that visibly surprises even him: he gave 5.6 his old fish slap game codebase and asked it to rebuild the game in 3D. Without being told anything about the game's theme or style, the model not only built a functional 3D game but also generated 3D assets using Blender via its CLI access. The resulting fish assets are recognizable if cursed, and the game itself plays and functions correctly. Theo hadn't read a line of the code. The moment lands as a real marker of how far things have come: a year ago the conversation was about whether models could code at all, now Theo's complaint is about the default hotkey choices. This segment also surfaces an interesting technical detail — Blender has a CLI, which the model discovered and used autonomously.\n\nA $200/month Codex subscription provides access to approximately $14,000 worth of API token compute in any given period according to Semianalysis measurements. With mid-month resets that restore a full week of usage, the practical ceiling hits around $20,000 per month — making extreme testing economically rational for power users.\n\nA $200/month Codex subscription can yield up to $20,000 worth of token usage per month when factoring in usage resets that restore a full week of compute.\n\nChapter 7 · 35:00\n\nLong-Running Tasks & What Actually Changed\n\nThis chapter traces the evolution of how both hosts actually use these models. Fable was the catalyst: it was the first model that made genuinely long, complex runs feel worth attempting. When Fable was taken away, they found 5.6 could carry those same expanded workflows further than any previous OpenAI model. Theo explains the economics: a $200 Codex subscription, per Semianalysis measurements, delivers up to $14,000 worth of compute per period — and with two usage resets that restore a full week, this can reach $20,000 per month[1]— Theo\"According to Semianalysis measurements, a $200/month Codex subscription can yield as much as $14,000 worth of token usage in a single perio…\"34:00. He's careful to note that the outrageous dollar figures on their dashboards ($131,700 and $93,000 respectively) represent deliberate stress-testing experiments, not practical work. His real workflow tasks — auditing open PRs, rebasing, coordinating merges — ran on a single 5.6 thread spawning targeted subagents, and were both cheaper and more valuable.\n\nBen's single run to port the Executor project to Rust and Svelte ran to 100 billion tokens and cost $65,000. The real driver wasn't the orchestrating reasoning model — it was the dozens of subagents it spun up underneath, which is the only way to blow through API usage this fast.\n\nBen's single longest run — porting the Executor project to Rust and Svelte — cost $65,000 worth of API tokens via xHI reasoning model orchestrating massive subagent swarms.\n\nTheo increased Codex's subagent limit from the default of ~3 to over 20 to enable more aggressive parallel work, which also pushed macOS to its limits.\n\nChapter 8 · 39:00\n\nWhy We Moved Agents to Linux Boxes\n\nThe move to Linux isn't ideological — it's pragmatic. Theo noticed macOS's syspolicyd consuming 215% of his M5 Max's CPU just from Codex running subagents in the background, more than rendering complex Blender scenes[1]— Theo\"Linux removes macOS process overhead: Moving agents to Linux eliminates macOS's aggressive process monitoring of Codex subagents, which was…\"39:15. Ben explains the root cause: MCP cannot share a single connection across threads, so 50 simultaneous subagents require 50 separate MCP server processes, 50 computer use instances, and all the associated overhead. macOS watches every single one of these processes suspiciously. On Linux, none of that overhead is present — you can spin up dozens of threads without touching system performance. Both hosts now run their long-running agent tasks on network-accessible Linux boxes, using T3 Code's web interface over Tailscale to monitor and interact with sessions from anywhere, including their phones.\n\nRunning 50 simultaneous Codex subagents on macOS means 50 separate MCP processes, causing syspolicyd to consume 215% of CPU on an M5 Max. Linux has none of these process-monitoring penalties, so you can spin up dozens of subagent threads simultaneously with no performance hit.\n\nMoving agents to Linux eliminates macOS's aggressive process monitoring of Codex subagents, which was causing syspolicyd to consume 215% CPU on an M5 Max.\n\nChapter 10 · 45:20\n\nCodex vs Claude Code: The Subagent Architecture Gap\n\nThis is the episode's most technically dense chapter, and arguably its most important. Theo carefully distinguishes between what a subagent is (a one-dimensional tool call that spins up an agent, waits for completion, and returns output) and what Claude Code's workflow primitive is (a vanilla JavaScript file the model writes itself, with dynamic stages, conditional subagent spawning, and true parallelism in a single tool call)[1]— Theo\"Claude Code gives agents a full JavaScript workflow file they write themselves — dynamic, multi-stage, and parallelizable in one tool call.…\"48:20. Codex's subagent system is a pre-built feature the OpenAI team coded; Claude Code's is code the agent generates on demand for the specific task at hand. This architectural difference, combined with Claude Code's superior TUI visibility into running subagents, has created a gap that both hosts now feel more acutely than the raw model capability difference between 5.6 and Fable. Ben adds that UltraCode in Claude Code simply adds a prompt layer pushing the model toward workflows, suggesting the gap is solvable but hasn't been solved yet in Codex.\n\nClaude Code gives agents a full JavaScript workflow file they write themselves — dynamic, multi-stage, and parallelizable in one tool call. Codex's subagent system is a pre-built feature agents work around. This architectural difference is now larger and more impactful than the raw capability gap between GPT-5.6 and Fable.\n\nBoth hosts concluded the capability gap between Codex and Claude Code's subagent/workflow systems is now larger and more impactful than the raw model capability gap between 5.6 and Fable.\n\nChapter 11 · 53:20\n\nModel Comparison Deep Dive: Log Analysis & Behavioral Differences\n\nTheo pulls out one of the episode's richest segments: two separate AI-generated analyses of his coding session logs, one comparing 5.5 to 5.6 and one comparing 5.6 to Fable-5. The 5.6 vs Fable comparison is the one that lands hardest. The analysis finds that Fable-5 'thinks wider' and is the stronger strategic advisor, while 5.6 'ships better' and is the stronger day-to-day coding agent[1]— Theo\"Fable 5 thinks wider, 5.6 ships better. Fable-5 is the stronger strategic advisor. F5-6 is the stronger day-to-day coding agent. The differ…\"54:40. The communication styles are starkly different: 5.6 produces terse, telegraphic build-bot receipts while Fable writes conversational prose that teaches the maintainer. But the most surprising finding is Fable's self-assessment blind spot: it voted for its own plan 6-0 in a head-to-head planning comparison, even while acknowledging benefits of the alternative[2]— Theo\"Fable is much worse at critiquing its own work. It thinks its outputs are gifts from heaven. Like it is the divinely mandated model that do…\"58:37. Ben interprets this as a feature of Claude's training — the 'Claude constitution' instills conviction that can become stubbornness — while 5.6's mechanical execution makes it genuinely neutral.\n\nRunning both models against the same session logs revealed fundamentally different identities: GPT-5.6 produces terse, telegraphic outputs like a build bot reporting to a coordinator. Fable-5 writes conversational prose that teaches the maintainer. Neither dominates every stage, but the behavioral divergence is accelerating.\n\nAn analysis of Theo's session logs comparing GPT-5.6 and Claude Fable-5 concluded Fable is the stronger strategic advisor while 5.6 is the stronger day-to-day coding agent.\n\nWhen tasked with comparing plans, Fable-5 voted for its own plan 6-0 even when it could acknowledge benefits of the alternative, showing it's worse at critiquing its own work.\n\nMCP cannot share processes across threads, so running 50 subagents simultaneously spawns 50 separate MCP server processes, which is what lights Macs on fire.\n\nChapter 12 · 1:00:20\n\nBurning Tokens, Long Runs & Practical Lessons\n\nThe hosts pull back from the extreme numbers and reflect on what actually moved the needle. The $65k Executor port was a meme — Ben admits it up front. But the days spent at $1,000 worth of tokens (about 1 billion) produced real, shipped work: Ben built a full Hermes agent interface system including a mobile app, web app, Mac Mini server, and Cloudflare tunnel architecture, all from a single coherent run. The key insight from both hosts is that the right mental model isn't 'do harder tasks' — it's 'go further in both directions.' Start earlier in the process, before you've even thought the problem through, and end later, after the agent has opened the app, tested its own work, responded to PR comments, and self-merged. These extended delegation loops are where the real leverage is found.\n\nBen spent approximately 1 billion tokens (around $1,000 at API pricing) to have 5.6 build a full Hermes agent interface including mobile app, web app, Mac Mini server, and Cloudflare tunnel.\n\nBen spent an entire night in deep back-and-forth dialogue with Claude Code to build an agents.md that embedded a specific, unusual worldview into his project context. The result: UI outputs with genuine personality that escaped generic LLM patterns. Out-of-distribution prompting produces out-of-distribution results.\n\n1:05:50\n\n1:08:40\n\nChapter 13 · 1:06:00\n\nGiving the Model Your Psychosis: agents.md Experiments\n\nThe episode takes its most memorable turn here as Ben confesses to a late-night experiment: he copy-pasted chunks of the Destiny universe's lore into a project's agents.md, ran the model with Groq subagents fighting each other alongside the main 5.6 instance, and got surprisingly good results. The UI outputs had genuine personality — custom SVG diagrams, interesting animations, and non-generic flow — precisely because the prompt was so far out of distribution[1]— Ben\"Ben spent an entire night in deep back-and-forth dialogue with Claude Code to build an agents.md that embedded a specific, unusual worldvie…\"1:05:50. He then took the concept seriously, spending an entire night in deep dialogue with Claude Code, sending five paragraphs back and forth until 9am, building a coherent internal tooling aesthetic he describes as 'cursed but functional.' Theo summarizes the philosophy: to escape generic LLM outputs, you need to give the model your psychosis[2]— Theo\"The thing he's trying to say is you need to give the model your psychosis.\"1:04:45. He closes the segment by joking that this is exactly how humanity gets wiped out — by showing the models Terminator to see how they behave.\n\nTo unlock 5.6's full potential, stop handing it tasks at the midpoint. Go two steps earlier in your workflow — let it think about the problem — and two steps later — have it spin up the app, adversarially review its own work, respond to PR comments, and merge itself. The model does all of it.\n\nClaude Code gives agents a full JavaScript workflow file they write themselves — dynamic, multi-stage, and parallelizable in one tool call. Codex's subagent system is a pre-built feature agents work around. This architectural difference is now larger and more impactful than the raw capability gap between GPT-5.6 and Fable.\n\nRunning both models against the same session logs revealed fundamentally different identities: GPT-5.6 produces terse, telegraphic outputs like a build bot reporting to a coordinator. Fable-5 writes conversational prose that teaches the maintainer. Neither dominates every stage, but the behavioral divergence is accelerating.\n\n54:05\n\n57:10\n\nSnapshots ()\n\nKey Quotes ()\n\nSign in to keep viewing\n\nCreate a free account to keep exploring this episode's insights, snapshots and quotes.\n\nA collaborator and tester referenced for sending screenshots of his GPT-5.5 frustrations and for building the T3 Code mobile app.\n\nAnthropic CEO jokingly referenced as someone who was removed from conversations, leading to improved progress on Fable's availability.\n\nThe AI lab that gave Theo and Ben early access to GPT-5.6 Sol; discussed in terms of model quality, naming strategy, and competitive positioning.\n\nDiscussed as OpenAI's primary competitor, credited with being ahead in model capability with Fable (Claude 4 Opus) and having better subagent tooling in Claude Code.\n\nUsed by Ben for Zero Trust access management and Cloudflare Tunnel to expose internal services securely from a Mac Mini.\n\nEpisode sponsor; a user management and authentication platform described as part of Ben's standard stack.\n\nEpisode sponsor; a localization/translation SDK used by Cursor, Ramp, and Netlify.\n\nResearch firm cited by Theo for measurements showing that a $200/month Codex subscription delivers up to $14,000 worth of compute.\n\nAnthropic's frontier model (referred to as Fable throughout), used as the primary benchmark against which GPT-5.6 Sol is evaluated; described as next-generation.\n\nOpenAI's coding agent product, compared unfavorably to Claude Code on subagent orchestration despite running on the more capable 5.6 model.\n\nAnthropic's CLI-based coding agent, discussed as having a significantly more powerful and flexible subagent/workflow architecture than OpenAI's Codex.\n\nThe predecessor to GPT-5.6, used as a regression benchmark; both hosts describe returning to it as painfully limiting after using 5.6.\n\nAn AI agent framework used by both hosts for personal automation tasks including email management, Discord notifications, and internal tooling.\n\nA coding agent interface product associated with Theo and Julius, providing a web UI for managing AI agent sessions across machines.\n\nAnthropic's Claude Opus 4.8 model, used by Theo to review and critique code written by GPT-5.6; it praised the architecture enthusiastically.\n\nTheo's software project, largely architected and built by GPT-5.6 Sol, used as a real-world test case for the model's capabilities.\n\nVPN mesh tool used by both hosts to access remote Linux coding boxes from their laptops and phones.\n\nAn open-source project that Ben attempted to port to Rust and Svelte in the $65,000 single loop run.\n\nStats\n\nEpisode stats\n\nInsight Overview\n\ninsights\n\nchapters\n\nInsight distribution\n\nSub-Categories\n\nSpeaker breakdown\n\nTalk Time\n\nThis episode\n\nClaims & Sources\n\n1 / 12 cited (8%)\n\nFactual claims made this episode, and whether a source was named.\n\n⚠\n\nTheo spent $131,700 in API tokens testing GPT-5.6 Sol during the early access period.\n\nTheono source cited\n\n⚠\n\nBen spent $93,000 in API tokens testing GPT-5.6 Sol during the early access period.\n\nBenno source cited\n\n✓\n\nA $200/month Codex subscription can yield up to $14,000 of actual compute usage per billing period.\n\nTheoSemianalysis\n\n⚠\n\nBen's single Executor port run cost $65,000 in API tokens and reached 100 billion tokens total.\n\nBenno source cited\n\n⚠\n\nsyspolicyd consumed 215% of CPU on Theo's M5 Max MacBook when running Codex with multiple subagents.\n\nTheono source cited\n\n⚠\n\nEach simultaneous subagent in Codex requires its own separate MCP server process because MCP cannot share connections between threads.\n\nBenno source cited\n\n⚠\n\nClaude Fable-5 voted for its own generated plan 6-0 in a head-to-head planning comparison, even while acknowledging benefits of the alternative plan.\n\nTheono source cited\n\n⚠\n\nGPT-5.6 Sol was tested by a significantly larger group of people before public release than after, making it possibly the first model release that reduced access at launch.\n\nTheono source cited\n\n⚠\n\nCodex's subagent limit defaults to approximately 3 subagents, which can be manually increased to 20 or more in settings.\n\nTheono source cited\n\n⚠\n\nClaude Code's workflow system lets agents write vanilla JavaScript orchestration scripts that can dynamically spawn multiple parallelized subagent stages in a single tool call.\n\nTheono source cited\n\n⚠\n\nAnthropic has never increased the price of an existing model tier; they only decrease prices, adding higher-priced new tiers for larger models instead.\n\nTheono source cited\n\n⚠\n\nAn AI analysis of Theo's coding session logs concluded that Fable-5 thinks wider and is the stronger strategic advisor, while GPT-5.6 ships better and is the stronger day-to-day coding agent.\n\nTheono source cited\n\nSign up free to see the full analytics\n\nCast, category & speaker breakdowns and fact-checked claims — free with an account.", "url": "https://wpnews.pro/news/we-tested-gpt-5-6-sol-early", "canonical_source": "https://vuci.ai/nerd-snipe-with-theo-and-ben/episode/we-tested-openais-56-sol-model-early/", "published_at": "2026-07-09 18:11:26+00:00", "updated_at": "2026-07-10 11:46:58.683314+00:00", "lang": "en", "topics": ["large-language-models", "ai-products", "ai-tools", "ai-agents", "ai-research"], "entities": ["OpenAI", "Anthropic", "GPT-5.6 Sol", "Claude Fable", "Claude Code", "Codex", "Theo", "Ben"], "alternates": {"html": "https://wpnews.pro/news/we-tested-gpt-5-6-sol-early", "markdown": "https://wpnews.pro/news/we-tested-gpt-5-6-sol-early.md", "text": "https://wpnews.pro/news/we-tested-gpt-5-6-sol-early.txt", "jsonld": "https://wpnews.pro/news/we-tested-gpt-5-6-sol-early.jsonld"}}