Import AI 479: Spacewar; low-latency tasks versus LLMs; Genie 4

A benchmark built by researchers at the University of Texas at Austin, Ohio State University, University of Illinois at Urbana-Champaign, NVIDIA, Northwestern University and Steadyworks found that frontier models struggle on real-time tasks, with GPT-6 Astra (high) posting the top pass rate of just 36% under a wall-clock setting on Realtime-Bench. The suite comprises 50 interactive tasks — 30 web tasks across travel, e-commerce, service and security, and 20 game tasks across arcade, puzzle, strategy and simulation — testing temporal state awareness, adaptive deliberation and time-aware action selection. Under a paused clock, models with high reasoning approach 70% to 80% success, indicating latency rather than capability drives the gap.

Import AI 479: Spacewar; low-latency tasks versus LLMs; Genie 4 Import AI https://importai.substack.com What do LLMs think about vision models? Welcome to Import AI, a newsletter about AI research. Import AI runs on arXiv, cappuccinos, and feedback from readers. If you’d like to support this, please subscribe. LLMs are very bad at low-latency tasks: …Realtime-Bench says it’ll be a while till AI systems can play games or do online auctions for you… Researchers have built Realtime-Bench, “a suite of carefully curated dynamic tasks where delayed actions directly affect success”. The test helps evaluate how well modern, general-purpose AI systems can perform tasks where decisions need to be made in real-time. The benchmark https://www.machinebrief.com/glossary/benchmark consists of 50 interactive game and web tasks, split across 30 web tasks in four categories travel, e-commerce, service, and security , and 20 game tasks arcade, puzzle, strategy, and simulation . Who did the research: The benchmark was built by people at the University of Texas at Austin, Ohio State University, University of Illinois at Urbana-Champaign, NVIDIA, Northwestern University, and Steadyworks. What it measures: The test is designed to probe for three qualities: temporal state awareness ”RT-Perceive” , where models can use observations to track changes in the world; adaptive deliberation ”RT-Deliberate” , where models alter their reasoning length and depth according to available time; and time-aware action selection ”RT-Act” where models need to appropriately time their actions for the situation. “For example, an agent purchasing a ticket must track the sale status and assess whether observed availability is still current RT-Perceive , limit further deliberation when the purchase window is closing RT-Deliberate , and choose between waiting for sales to open and purchasing while tickets remain available RT-Act .” They have three different conditions: Wall clock, which is designed to test out performance under real-world latency. Paused clock, which lets you arbitrarily pause things so you can get a sense of baseline capability, and Token clock, which “advances the environment at a fixed rate per generation token. An action is applied at the simulated time reached when it is committed, so consuming more tokens delays execution regardless of actual inference https://www.machinebrief.com/glossary/inference latency”. How well do models do? The authors tested out models from OpenAI GPT-6-Astra, GPT-5.6 Sol, GPT-Realtime-2, OpenAI Decisions , Anthropic https://www.machinebrief.com/glossary/anthropic Claude https://www.machinebrief.com/compare/claude-4-opus-vs-gpt-o3 Fable 5.1, Claude Opus 5 , Google Gemini https://www.machinebrief.com/compare/gpt-4o-vs-gemini-2-pro 3.8 Flash, Gemini 3.1 Flash Live, Gemini 3.8 Live , Jev 1.13, as well as Chinese models from Deepseek https://www.machinebrief.com/compare/llama-4-vs-deepseek-r1 V4.1 and Moonshot AI Kimi-K3 . GPT-6 Astra high got the highest pass rate on the ‘wall clock’ evaluation https://www.machinebrief.com/glossary/evaluation setting, with 36%. By comparison, with a paused clock, models with high reasoning get closer to 70% or even 80% success. Why this matters: This benchmark is an effective proxy for where AI agents will break - the low scores here point to tasks where AI agents cannot yet be expected to perform well. Check out the leaderboard : Realtime-Bench official website https://realtimeagents.org/ . Read the paper : Realtime-Bench: Can Frontier Models Solve Real-time Interactive Problems? arXiv https://arxiv.org/abs/2610.22155 . The future of spacewar, courtesy of the Defense Intelligence Agency: …Anti-satellite tech, space nukes, and more… Working in AI can be stressful so sometimes to relax myself I like to read about other topics to get a sense for what horrors lie elsewhere. I take the same approach with depression, where if I feel sad I will try and listen to music by people sadder than me in an attempt to cheer myself up e.g., Radiohead, Neil Young, and in very dark moments Bill Callahan / Smog, etc . So with that in mind I recently read “Challenges to Security in Space”, by the Defense Intelligence Agency. What’s going on in space: As with conflict on Earth, space is defined by conflict between the US versus China, Russia, Iran, and North Korea, with China the most technologically sophisticated, and Russia the most seemingly gung ho. “China and Russia are developing a formidable spectrum of offensive capabilities to degrade or destroy space services of other countries, from kinetic destruction to sophisticated jamming and cyber operations,” the DIA writes. “China has more than tripled its space launches per year in the past decade, with 22 launches in 2016 and 93 launches in 2025… Beijing judges space is central to the United States and its allies’ operations. Beijing also considers space an offensive military domain, which has shaped integral components of PLA military planning and campaigns.” Spacewar: The general vibe you get from reading this 100+ page report is there is a huge amount of ugly, frightening stuff happening in space, or being done to things in space by things on the ground. Some example quotes: - “By the late 2020s, China may field space-based directed energy systems and higher-powered ground-based directed energy systems capable of threatening satellites in higher orbits” - “China is developing space-based technology, including on-orbit counterspace weapons, capable of holding space systems at risk” - “Russia’s Peresvet, a ground-based laser system, is designed to dazzle enemy satellites’ optical systems to mask the movement of Russia’s strategic missile systems” - “Russia has launched several satellites over the last few years which conducted test activities on orbit for technologies that could enable attacks on other countries’ satellites and result in temporary or permanent damage” - China has a couple of spaceplanes, akin to the US’s hyper-secret X-37B vehicle, and they appear to be getting more sophisticated. In 2023, one of China’s aforementioned vehicles, “released and interacted with an object.” Satellite-on-satellite fights: It’s obvious throughout the report that we’re in a world where rival nations have satellites that try to go up and mess with other satellites from other people. - “In 2019, Russia launched satellite Cosmos 2542, which released a second satellite, Cosmos 2543. Cosmos 2543 then began following a U.S. national security satellite and approached it multiple times at a distance close enough to create potentially dangerous operating conditions,” the DIA writes. “Russia is developing kinetic orbital systems to permanently disable, or “kill,” satellites. Moscow’s Burevestnik is a developmental, co-orbital air-launched ASAT weapon for killing spacecraft in LEO”. - “In February 2022, Russia launched a satellite called Cosmos, which Moscow claimed at the time was testing the effects of space radiation on electronic systems. This satellite was launched into an unusual and isolated orbit not used by any other spacecraft”. Space nukes: Russia is experimenting with nukes in space. “A nuclear detonation in space would create an electromagnetic pulse, which would immediately affect satellites within range and could impact civilian infrastructure,” the DIA writes. “A nuclear detonation in space could be tailored to impact satellites in most orbits—LEO for commercial and reconnaissance satellites, GEO for critical communications, and MEO for navigation and timing satellites”. Why this matters - infrastructure for the next phase of the AI power struggle: As people spend more resources on building space-based data centers and eventually, perhaps, lunar-based ones , space will become ever more strategically and commercially valuable and ever more contested. Research like this gives us a sense of the hardware which is already in space and will arrive in the future. Perhaps in some far-off future an AI might seek to take control of such space assets as well. Read more: DIA Releases Challenges to Security in Space Defense Intelligence Agency, press release https://www.dia.mil/Articles/Press-Release/Article/4633953/dia-releases-challenges-to-security-in-space/ . Read the report : Challenges to Security in Space DIA, PDF https://www.dia.mil/Portals/110/Images/News/Military Powers Publications/Challenges Security Space 2026.pdf . Metis: A startup trying to fuse symbolic and neural reasoning: …Towards Vulcan-like machine minds… Metis, a startup that wants to build novel AI architectures oriented around developing machines with better reasoning, has published a research paper laying out its research approach. The high-level summary is simple - Metis wants to do for modern open-domain problems what DeepMind did for closed-domain problems like board games via its work on AlphaGo and AlphaGo Zero. What they want to do: The key idea here is to add a reasoning core to modern AI architectures which is far more reliable than what we have today. “In modern AI architectures much of the other surrounding machinery that made AlphaGo a successful reasoning system remains either implicit or absent,” they write. “The first missing element is an explicit and persistent epistemic state… the second missing element is a criterion that allows the system to choose reasoning moves that make progress towards the goal.” This is more of a position paper than a technical paper, so they don’t have a ton of detail here. The main idea is to train a reasoning policy and value function based on real-world data collection, then use this to guide the system to perform reasoning that reduces long-term uncertainty, and have these sit on a basic core of data made up of claims or hypotheses about the world. “Any change to a claim’s evidential status must be licensed by evidence or verified through explicit calculation and approved by an independent judgment system,” they write. “Model-generated content cannot enter the accepted epistemic state merely by being plausible; it must pass warrant assessment, which lowers hallucinations. Sources of evidence can be attached directly to the propositions they support, providing a clear chain of provenance. The reasoning process becomes explicitly auditable.” Why this matters - building reliable substrates that are also deeply inhuman: Ideas like the architecture described here are fascinating both because of the promise they bring for increasing the reliability of AI systems, but also for how deeply inhuman they are: humans do not reason from some pristine base of facts, rather it’s all a big messy blob of memory and intuitions and some rules some of which are learned from experience and some of which seem more platonic, like forms of logical reasoning . In this way, contemporary LLMs are more like humans than the system described here. “While the history of AI can be understood as a contest between symbolic and neural methods, it may be more constructive to view it as a story of two incomplete successes,” Metis writes. “Classical systems made aspects of the reasoning process explicit but depended on representations, operators, heuristics and knowledge supplied largely by hand. Contemporary LLMs learn extraordinary semantic operations and vast bodies of implicit knowledge but leave the organisation, evaluation and accumulation of reasoning implicit.” The AI system described here would ‘think’ in a way that is very different to how humans think and such a system might show us more about the differences between our own intelligence and that of machines. Read more: The Age of Reasoning Metis, PDF https://metisreasoning.ai/whitepaper/the-age-of-reasoning-2026-10.pdf . Google summons Genie 4 for better world modeling: …One of Google’s most interesting projects continues to live ... Google DeepMind has published details on Genie 4, a new iteration of its Genie world model https://www.machinebrief.com/glossary/world-model . The company is rolling out access to Genie 4 currently as a research preview for some trusted testers and will eventually come to Google AI Ultra subscribers. The Genie series of models are fascinating, showing a world where you can take a photograph of anything and turn it into an interactable environment, or one where you can convert a 2D drawing into the stage for a videogame, and more. Key features: Google calls out four features that make Genie 4 different to previous ones. These include: - Real-time modification : “You can use natural language to modify a simulation in real-time — like changing the weather, moving the camera, or introducing a new person or object — and Gemini will contextually adapt your prompt to match the surrounding scene.” - Physics and human action simulation: “Genie 4 simulates real-world physics scenarios, such as aerodynamics, collisions, and more,” and it also simulates people better. “The most dramatic leap is how Genie 4 simulates people, creating characters that move naturally, respond organically to their environments, and can be interactively steered in real-time.” - Sound modeling: “Genie 4 is our first real-time world model to generate audio natively,” Google writes. It also “introduces interactive dialogue that adapts on the fly.” - Simulating complex events : Google has embedded Gemini 3.8 Flash into Genie 4, so that the fast Flash model “serves as a guiding agent in the loop”, providing continual calibration. “Gemini continuously observes the scene, breaks down goals into a sequence of steps, and reviews incoming frames to track the progress and verify that each step succeeded. As the world shifts or the user intervenes, Gemini adapts its plan in the moment, unlocking worlds with reliable causality, procedural behaviors, and multi-step reasoning.” Why this matters - towards a multitude of worlds all dreamed up by machines: Though the technology is currently quite janky e.g., robotic voices, a dreamlike sort of inconsistency in the visualized environments, etc , it points towards a world where we can generate infinite forms of entertainment and potentially data? via creating and prompting custom worlds. Read the announcement post : Genie 4: A new generation of world simulation Google DeepMind blog https://deepmind.google/blog/genie-4-a-new-generation-of-world-simulation/ . Watch the video : Genie 4: A new generation of world simulation Google DeepMind, YouTube https://www.youtube.com/watch?v=sFfPkUJd Jk&t=5s . Tech Tales:City Race Description of a game played in the 2040s One of the favorite games that children would play after The Uplift was called City Race; in it, they competed against one another to plan, build, and manage cities, expanding across the domain of a novel planet. The cities were trapped in time, starting with the technology of whatever era they were in, and the games took place over the course of about a century, allowing some further technology development to occur. Each game was played using a slice of the resources from the Dyson Swarm, allowing for a powerfully complex simulation albeit with local-agent cognitive gating, such that nothing could develop enough sophistication to be deemed a Conscious Entity . Hundreds of thousands of simulated minds lived and died, sometimes thriving in cities run by careful and effective owners, and other times trapped in squalor, or growing old in places whose economies were hollowing out; casualties of the successes of others. It was deemed good for the children to learn what it meant to be gods, before they became adults. At the closure of every game, the children would need to spend a day living inside each of the cities in the gameworld they had played; those who had built marvelous cities experienced the wonders of their own creation, then walked through the broken towns whose success had been stymied by their own. Those who had struggled lived in their twilight towns, then got to tour the far off places that had doomed them. Things that inspired this story: Player of Games by Iain M Banks; Simcity; game playing; urban design; The Glass Bead Game by Hermann Hesse. Thanks for reading Get AI news in your inbox Daily digest of what matters in AI.