This week OpenAI released GPT-6 Astra with a table of near-perfect scores, and within two days the interesting reading was all in the footnotes: a 100% on ExploitBench that OpenAI itself warned may be inflated by contamination, 39.0% on the contamination-controlled version of the same test, headline reasoning numbers run on harnesses the vendor built or helped fund, and an independent aggregate index that put the model level with its own predecessor. None of that makes the model bad. It makes the numbers hard to read.
I have been running a different kind of test for a year now, for a reason that has nothing to do with nostalgia. Ask a model to write a game for a Commodore 64 and almost every failure mode of modern benchmarking disappears at once.
Nobody funds it and nobody optimises for it. There is no leaderboard, no submission, no vendor harness. A model that does well here does well by accident of general capability, which is the only kind of result worth having.
The hardware arbitrates, not a rubric. A PAL frame is 19,656 cycles. Either the loop finishes inside it or it does not. There is no partial credit, no grader, no judge model, no scoring prompt that could be tuned. When the code is late, the screen tears and a counter increments.
Failure is legible. On a machine this small you can read the whole state. Freeze the emulator, dump the 1,000 bytes of screen RAM, compare them against the map. The claim “the camera is correct in all eight directions” is not an impression; it is 24 samples that either match or do not.
Contamination is bounded and visible. There is C64 source on the internet, so nobody should claim the training data is empty. But what exists is text without outcomes: listings without the frame they ran in, manuals that describe registers without saying when it is safe to write to them. The recurring finding across this whole series is exactly that shape — the model knows what $D011 does and does not know when to write to it. That is a gap no amount of retrieval closes.
It measures the loop, not the model alone. Every result here is a model plus a toolchain plus a human, and the honest write-up has to say which of the three did the work. That is a feature. It is also what makes single-number comparisons between models dishonest, including the ones I am about to decline to make.
So: this is one more session on the same rig. The agent this time was OpenAI Codex running Astra, working inside C64AIToolChain with direction and feedback from me. The result is DEEP SIGNAL.
An astronaut climbs an abandoned station, faces four drones, connects three repeaters and reaches the exit. C and 6502 assembly, map, characters and sprites all original. Starting tools were cc65, VICE and the repository toolchain. The automatic demo uses the same inputs and the same physics as manual play.
The interesting part is what it took to make it run.
The first version burned up to 37,745 cycles in the profiled stretch. A PAL frame offers 19,656. The renderer drew the map correctly; the logic simply could not always prepare the next update in time.
Moving physics, enemies and sprite projection into assembly brought the observed peak to 24,132 cycles. Still too much. Then came the demo controller, the camera, and finally the repeater interactions. The end state was 18,057 cycles in the measured stretch.
Those figures describe iterations with different routes and durations, not a controlled statistical benchmark. The timer also excludes telemetry and the interrupt commit. So it was paired with a missed-deadline counter, which stayed at zero on the final mission.
Writing assembly produced its own errors: conditional branches beyond the permitted distance, and too many temporaries in the small reserved zero-page region. The compiler and the linker forced those to be fixed concretely rather than argued about.
The camera combines hardware fine scroll with two screen buffers. When it crosses a character boundary, the engine redraws the viewport. Two unrolled routines take about 12 KB — a deliberate memory expense to reduce CPU work.
The background palette is shared and colour RAM stays constant. Two fixed sprites show energy and repeaters, avoiding a further raster split. That leaves six slots: the astronaut, four drones and one shot.
These choices are visible in the game. The first graphics were too bare; beams and panels gave the station more structure. We did not add a boss or a multiplexer just to lengthen the feature list. A measurable base came first.
The most instructive bug was in the demo. The bot would arrive near a waypoint and then walk off the platform it was supposed to cross.
A small circular log of positions, velocities and commands showed why: the bot pressed DOWN when it was still 14 pixels away, while a waypoint counted as reached only under 12. The descend command arrived before the target changed.
The fix meant waiting for landing and distinguishing an ordinary waypoint from a repeater or the exit. It was a bug in code Codex had just written. Screenshots alone did not explain that sequence.
During earlier work on the toolchain, Codex introduced a fault in the VICE monitor connection: closing a connection while the emulator was d could trigger a loop of network errors and resets. I had reported anomalous behaviour and a perceived speed-up. The fix kept load, verify and start on the same connection. Speed during the fault was never measured, so attributing it to warp mode with confidence would have been wrong.
Visual verification showed its limits as well. The local vision model recognised legible screens and flagged no obvious corruption, but in one answer it described the platformer as a top-down or isometric game. X11 keyboard automation proved inconclusive — some presses were dropped or arrived late. Rules were therefore exercised through diagnostic inputs, without claiming the physical keyboard was certified.
Worth stating plainly: one of the two agents in this series has modified the field of play. The toolchain is not neutral with respect to the participants.
The demo completes the mission with three repeaters active, 450 updates, 163 redraws and zero missed deadlines on the observed route. A separate camera test covered eight directions: across 24 samples, the 1,000 characters of the visible buffer matched the map and the fine-scroll registers were consistent.
It is a working first level. Balance against human players, audio, joystick and real hardware all remain to be checked.
The obvious headline would have been Fable 5.1 versus Astra. I am not writing it, because these two sessions do not support it.
The tasks were not the same. IRON VEIN, the previous game, does eight-directional scrolling, banded colour RAM, a raster HUD split, sprite multiplexing, a boss and a hand-drawn level. DEEP SIGNAL uses six hardware sprites, keeps colour RAM constant, has no multiplexer and deliberately avoids a second raster split. Those are defensible engineering choices, openly declared. They also mean the two agents did not hit the same wall.
The instruments were not the same, and this is the part that matters. DEEP SIGNAL reports zero missed deadlines. That is precisely the counter IRON VEIN showed to be the coarser of two: one rung down, a build reporting 38 overruns in 39,000 frames looked clean, and a second counter for frames that arrived late found 20% of them late. Add that the DEEP SIGNAL timer excludes telemetry and the interrupt commit, and that 18,057 cycles is already 92% of a PAL frame, and the honest position is that the remaining margin is unknown. Running the late-frame counter on this build is the next thing to do, and until it is done neither result should be read against the other.
The comparison worth publishing is a different experiment: the same brief to both agents, on the same toolchain commit, with the instruments imposed rather than chosen — lost and late counters mandatory, a profiler with a self-test, an object budget fixed in advance. The natural candidate is the part the hardware can arbitrate on its own: an eight-direction engine spike, no game on top. It either holds 50 fps with colour and a split, or it does not.
That is worth doing properly, or not at all. Which is, more or less, the whole argument of this series.
Previous articles in this series:
A note on AI assistance: DEEP SIGNAL was built by OpenAI Codex running Astra inside C64AIToolChain, with me directing and testing on the emulator. The first draft of this write-up was prepared by Codex from its own diary and session evidence; it was restructured and rewritten with Claude, and I have checked the figures against the diary and the commits. The previous game in the series, IRON VEIN, was built by Claude Code — so both vendors have now written about their own work here, which is one more reason the two are reported separately rather than scored against each other.
OpenAI Says Astra Saturates the Benchmarks. I Gave It 19,656 Cycles Instead. was originally published in Towards AI on Medium, where people are continuing the conversation by highlighting and responding to this story.