# Latency Is Now the Bottleneck > Source: > Published: 2026-09-02 04:00:00+00:00 # Latency Is Now the Bottleneck Patrick Collison recently noted that AI agents are still stuck in Terminal UIs: I love agentic coding harnesses, but they shouldn't be primarily terminal-based. The terminal is great for quick and precise commands, but information density is extremely low and UI affordances are minimal. Maybe provision of TUIs is worthwhile for occasional use (when… — Patrick Collison (@patrickc)[August 16, 2026] I tend to agree, [as do others](https://sockpuppet.org/blog/2026/08/20/stop-making-tuis/). But why is this the case? Why haven’t we gotten past the [MS-DOS era of AI Interfaces](https://x.com/ekuyda/status/2026759319467995209)? While the models are incredibly smart, they are *so slow*. If we want real “AI Intefaces”, we need models that are smart *and fast*. Low latency is critical for interactive applications! I’m inspired by Krea’s [Realtime Edit](https://www.krea.ai/blog/realtime-edit), where an image is generated while you draw or prompt, seemingly instantly: introducing Realtime Edit. — Krea (@krea_ai) now, like with Nano Banana, you can edit images with complex instructions in real time. join the beta 👇[pic.twitter.com/fTxSeCAoZT][January 22, 2026] Why can’t the rest of our “AI apps” feel this interactive? Perhaps they can, and soon! Here’s a rough prototype of what “low-latency” AI Interfaces could look like in other domains. This is not faked nor sped up: Designing with AI should feel instant. — smiz (@__smiz) Claude generated a landing page, and here's me making a few edits. (the gradient change could have been better)[pic.twitter.com/i9UcEkVVq7][September 1, 2026] Remember that, like in the past, this is the dumbest and slowest the models will be. It’ll only get better from here. ## Smart Enough; Then, Fast Enough A few years ago, the bottleneck for AI was intelligence. GPT-3 simply wasn’t smart enough to complete compelling tasks. We’ve since hill-climbed that gap over 6 years, and with reasoning models we can accomplish incredible tasks at the cost of very, very long latency. Some models end up working for over a day… a fact that AI labs like to brag about! Even if we can [approximate faster feedback with Luna](/checkpoints/pi-duplex), it’s still much too slow for interactions like click, drag, etc. If we want snappy AI interfaces that feel like magic, **we need AI models to complete responses within 100ms** 1. That sounds hard, but I’ve learned not to doubt the number of talented, motivated researchers out there. ## Can Autoregressive Transformers Get Us There? Astute observers will question whether the demos above used the Mighty Autoregressive Transformer… and they’d be correct. The “real-time” models are, in fact, [Diffusion models](https://huggingface.co/blog/Esmail-AGumaan/diffusion-models). In my demo, I used a Diffusion Large Language Model (dLLM) from Inception Labs: [Mercury 2](https://www.inceptionlabs.ai/blog/mercury-2-the-first-reasoning-model-fast-enough-to-pick-up-the-phone). It was the best diffusion text model I could find that had an API, since I would’ve had to host [DiffusionGemma](https://deepmind.google/models/gemma/diffusiongemma/). The hesitancy around Diffusion models is that historically they haven’t been as smart as LLMs, and that they’ve been tricky to post-train. Fortunately, the outlook has changed a lot in the last year. Kaiming He - Mr. [ResNet](https://arxiv.org/abs/1512.03385) himself - is [back](https://scholar.google.com/citations?user=DhtAFkwAAAAJ&hl=en), publishing [bangers](https://openaccess.thecvf.com/content/CVPR2026/papers/Li_Back_to_Basics_Let_Denoising_Generative_Models_Denoise_CVPR_2026_paper.pdf) [about](https://arxiv.org/pdf/2607.09024) [diffusion](https://arxiv.org/pdf/2605.10938). Some of his work [piques my curiosity](https://arxiv.org/abs/2602.04770), but it sounds like a big focus will be engineering work to train diffusion text models at scale. [Nicholas Boffi](https://nmboffi.github.io)’s group has also quite bit on [diffusion text models](https://one-step-lm.github.io/blog/index.html) and [distillation](https://arxiv.org/abs/2512.02636). Here’s a more comprehensive research recap for diffusion models by Sander Dieleman: [“Continuous diffusion language models”](https://sander.ai/2026/08/24/continuous-dlms.html) ## How Will It Shake Out? Taking it all in, I think the following is sensible to expect: - Smart + Real-time AI is not only tractable but inevitable. Once generally available, we’ll hit a new era of “AI Interfaces” that will make 2026 look like the ancient past. - I don’t know if Diffusion or [Drifting](https://arxiv.org/abs/2602.04770)or something else is the answer. But it is obvious that autoregressive models can’t be used for the AI Interfaces we want. However, they will be used for “long-horizon” tasks where intelligence is table-stakes, and certainly for labeling. - Outside of Deepmind, none of the AI labs are seriously focused on dLLMs. This means that a [neolab can take the lead](https://x.com/coreauto/status/2092363687550161322), assuming they are not acquired beforehand. Strap-in, folks. It’s only just the beginning. ## Footnotes - In [“How Much Faster Is Fast Enough”](https://dl.acm.org/doi/10.1145/2702123.2702300), direct and indirect taps have JND’s around 90-100ms. This seems like a nice proxy for “magical, instant” feedback driven by AI, but happy to look at other papers.[↩](#user-content-fnref-1)