The pelican benchmark is doubtful, let's draw MacBook Pro in SVG. Is Fable best?

A new benchmark replaces the pelican SVG test for AI models, asking them to draw a MacBook Pro 16 in SVG. All nine frontier models passed the pelican test on first try, indicating it no longer differentiates performance. The MacBook test provides precise ground truth and a difficulty ladder, with Claude Fable 5 producing the most detailed drawing at $0.73 and Gemini 3 Flash the simplest for $0.007.

TL;DR - All 9 frontier models pass the pelican first try. Not because they all got brilliant - the pelican is the most famous drawing prompt in AI, it is all over the training data, and it never had a precise ground truth to grade against. - Our replacement: draw a MacBook Pro 16 in SVG, one shot. Everyone knows the exact proportions from memory, it has dozens of parts, and the 3D view adds perspective - plenty of ways to fail visibly. - Claude Fable 5 at xhigh effort drew the best MacBook $0.73 per drawing - clearly more detailed than everything else. GPT-5.6 Sol at xhigh is second. Gemini 3 Flash drew the simplest coherent one for $0.007. - The effort dial peaks at xhigh : on the strongest models it buys visible quality, then max falls off a cliff - three models never finished the drawing and one hung for 33 minutes. - Same task, ~170x price spread between the winner and the cheapest passable drawing. For two years, the best quick test of a new AI model was Simon Willison's pelican https://simonwillison.net/tags/pelican-riding-a-bicycle/ : ask the model to draw a pelican riding a bicycle as SVG and look at the result. It worked because it could not be faked - the model either understood shapes, spatial relations, and SVG, or you got a white blob on two circles. It no longer separates anyone. We ran the pelican across today's frontier models and they all pass. So we built the test we actually wanted: draw a MacBook Pro 16 in SVG, in a dark color, one shot . Everyone reading this knows exactly what a MacBook Pro looks like, so everyone reading this can judge the results. 9 models, 3 reasoning-effort levels, 23 runs, $4.58 in API costs , zero retries, zero cherry-picking. The gallery below is raw model output. What happened to the pelican A benchmark is useful exactly as long as it separates models. The pelican was brutal in 2024 and informative in 2025. It did not die - it has two specific problems now, and we wanted to show them rather than assert them, so we ran the pelican as a control through the identical harness: same nine models, same effort ladder, same one-shot rule. Here is every model's first-try pelican at standard effort: Every single one is a pelican, on a bicycle, first try. When every student gets an A, the exam has stopped grading. The numbers say the same thing more sharply: the models spend roughly 4x fewer output tokens on a pelican than on the MacBook you will see below Opus 4.6: 2,091 vs 8,156 , the whole 23-cell pelican matrix cost half as much $2.24 vs $4.58 , and cranking reasoning effort barely moves anything - Opus 4.8's pelican costs $0.03 at high and $0.04 at max and looks the same. The task no longer makes anyone work. Two problems got it here: Everyone has trained on it. The pelican is the most famous drawing prompt in AI. Every model release gets its pelican posted publicly; three years of pelicans and commentary about them are in every training corpus, and every lab knows reviewers will run it on day one. At this point a good pelican cannot tell you whether the model understands geometry or has simply seen ten thousand graded pelicans. It never had a gradation. Nobody knows exactly how long a pelican's beak should be relative to its wingspan, so grading tops out at "recognizable" - and everything above that is connoisseurship, not measurement. There is no scale on which one passing pelican beats another. What made it great in the first place still holds - you know the ground truth without a rubric, and style cannot hide missing parts. The successor just needs both problems fixed. A MacBook Pro fixes them: hundreds of millions of people have stared at one for thousands of hours, so the proportions of the lid, the keyboard grid, the notch, the hinge, and the trackpad have a precise shared ground truth - your eye flags a 5% error instantly. And the task has a real difficulty ladder: dozens of parts plus a 3D perspective view, which is much harder to keep consistent than a flat side-on pelican. Models can now fail at many different heights instead of clustering at "recognizable" - which is exactly what a benchmark is for. The setup The prompt, in full: Create an SVG 3D model of a MacBook Pro 16 in a dark color. Respond with ONLY the SVG markup, no explanation. One API call per cell. No retries, no system prompt, no examples, no cherry-picking - the first answer is the answer. We ran each of the top models at three reasoning-effort levels high , xhigh , max , because effort dials are the most expensive knob in 2026 pricing and nobody publishes what they actually buy you. Grok 4.5 runs at high only that is its ceiling . Gemini 3 Flash runs at its default - it is the cheap-and-fast control. Costs are the providers' own usage-reported billing, not estimates. The gallery: 23 runs, unedited Every image below is inline SVG, byte-for-byte what the model returned. Open your browser's view-source if you want to audit any of them. Effort levels marked with an asterisk were requested as xhigh and clamped to high : the Claude 4.6 generation rejects xhigh outright more on that below . The verdicts Ranked by what the drawings actually look like click any cell above to judge full-size : Claude Fable 5 at xhigh - the best drawing, clearly the most detailed GPT-5.6 Sol at xhigh - a genuinely good laptop, loosely a MacBook Gemini 3 Flash - simplest, but nothing is broken Fable 5 / Opus at high - one tier: recognizable, rough around the edges Sonnets, Grok 4.5 - broken geometry, or barely trying The winning drawing is Fable 5 at xhigh $0.73 : a dark, correctly assembled MacBook where the extra effort shows up as detail - a believable screen panel, keyboard grid, hinge, and lighting, with every part still in the right place. At plain high , Fable drops to roughly Opus's tier: the same class of drawing with nicer lines. On this model the effort dial buys real quality, not decoration. Sol attempted more than anyone - more gradients, more perspective, more parts. At high that ambition bends the geometry: panels misalign, the perspective fights itself. At xhigh it pulls most of it together into a genuinely good laptop - admittedly a bit more dark Lenovo than MacBook, but detailed, coherent, and a clear second place. The result we did not expect came from Gemini 3 Flash. It drew a simplified laptop - fewer parts, flat shading - and it holds together: nothing is broken, nothing floats, it reads as a laptop at a glance. For $0.007 and 14 seconds, that is a striking price-to-coherence ratio. It seems to know what it can execute and stays inside that envelope. Everything else falls apart under inspection. Opus and Sonnet produce laptops with misplaced parts and broken geometry; Grok's minimal attempt 668 output tokens - it barely tried is more logo than laptop. On a pelican, most of these models look fine. On an object you know intimately, the differences are unmissable. The effort dial peaks at xhigh - then falls off a cliff The effort ladder was supposed to be the boring part. It produced the sharpest result of the benchmark: effort is not monotonic. The curve rises, peaks, and then destroys the run. Up to Fable's win and Sol's second place both happen at xhigh , effort buys real quality on the strongest models. xhigh , and the improvement over high is visible detail, not polish. On mid models it buys little - Opus 4.8's three drawings cost $0.05, $0.10, and $0.18 and look like the same drawing. At Fable 5, Sonnet 5, and Terra reasoned so long they consumed the entire 16,000-token output budget before closing the SVG. You pay the highest price on the ladder and receive a truncated file. max , three models never finished the drawing. GPT-5.6 Sol at We stopped waiting after 33 minutes. max did not return at all. On a bounded creative task, reasoning effort has a peak, and it is not the top setting: past it, the model spends its budget reasoning about the drawing rather than producing it. If you operate an AI product with an effort dial, this is worth measuring on your own workload before you let users pay for max . The benchmark found a real bug on its first run A side-effect worth disclosing: the first full run of this benchmark crashed two cells with an API error we had never seen in production - This model does not support effort level 'xhigh' . It turns out Anthropic's xhigh effort exists only on the newest generation Sonnet 5, Opus 4.8, Fable 5 ; Opus 4.6 and Sonnet 4.6 reject it, and their supported list low, medium, high, max is not even a superset-consistent scale across generations. Our product maps a user-facing quality dial to these levels, which means one combination of settings produced a hard error on every message. The benchmark surfaced it in ninety seconds; we shipped the fix the same night. A static leaderboard would never have caught it. What one drawing costs: a 200x spread Because every cell is a real API call, we can put a price on the same task across the market: | Model · effort | Cost per drawing | vs cheapest | |---|---|---| | Grok 4.5 · high | $0.004 | 1x | | Gemini 3 Flash · default | $0.007 | 1.6x | | Claude Sonnet 5 · high | $0.03 | 6x | | Claude Opus 4.8 · high | $0.05 | 12x | | GPT-5.6 Terra · high | $0.09 | 20x | | GPT-5.6 Sol · high | $0.26 | 60x | | Claude Fable 5 · high | $0.27 | 63x | | GPT-5.6 Sol · xhigh 2nd place | $0.45 | 105x | | Claude Fable 5 · xhigh the winner | $0.73 | 169x | | Claude Fable 5 · max | $0.80 truncated | 186x | The winning drawing cost about 170 times the cheapest entry, and the cheapest entries were not embarrassing. Whether Fable's detail is worth 100x Gemini's coherence depends entirely on what you are building - but you cannot even ask that question from a $/Mtok pricing page. Why the sticker price misleads across vendors is its own story: we measured it in The Same TypeScript Costs 73% More on Claude Than on GPT /blog/real-price-of-frontier-models . Try it yourself The method is deliberately reproducible: one prompt, one shot, providers' own usage numbers. The prompt again, ready to paste into any model: Create an SVG 3D model of a MacBook Pro 16 in a dark color. Respond with ONLY the SVG markup, no explanation. Or swap in any object your audience knows intimately - a Coke can, a Vespa, a Stratocaster. The requirements are only that the ground truth lives in your reader's head and that style cannot hide broken geometry. We will re-run this gallery when the next frontier models ship. The pelican had a good run; the MacBook has years of discrimination left in it - the day every model draws it correctly, we will happily retire it too. Method notes: all runs 2026-07-13/14 via each provider's public API Anthropic Messages, OpenAI Responses, xAI and Google chat completions , temperature and other sampling left at provider defaults, output capped at 16,000 tokens, reasoning effort set via each provider's native parameter Anthropic output config.effort , OpenAI reasoning.effort , xAI reasoning effort . Exact model IDs: claude-fable-5 , claude-opus-4-8 , claude-opus-4-6 , claude-sonnet-5 , claude-sonnet-4-6 , gpt-5.6-sol , gpt-5.6-terra , grok-4.5 , and gemini-3-flash-preview note: a preview build - the only non-GA model in the set . Costs are computed from the providers' own usage-reported token counts at list prices. SVGs are unedited; failure cells are reported as failures rather than re-rolled. The measurements are ours; the prose was drafted with AI assistance and edited by a human. Playcode keeps all of these models one click apart, with the same effort dial we benchmarked here - so you can run your own MacBook test on a real project instead of taking our word for it. Try it at playcode.io https://playcode.io .