# Reka's Rho-1: What Happens When One Model Replaces Your Multimodal Pipeline > Source: > Published: 2026-10-06 09:38:27+00:00 Most "multimodal" AI products today are really a relay team. One model plans, then hands the job to an image model, a video model, a detector, or a robot policy. Reka's new **Rho-1** research preview asks a blunt question: what if there were no handoffs at all? ## What was announced (Oct 5, 2026) - **One 19B model, trained from scratch.** Rho-1 understands and generates text, images, and video, reasons over them, and emits robot actions, all inside a single neural network. - **One shared state.** Text, image latents, video frames, and robot actions all live as tokens in the same context window and the same KV cache. Reka says there is no tool call and no second model in its demos. - **Two token types.** Discrete tokens carry text and commands. Continuous tokens carry image and video latents, robot actions, and proprioception, so rich signals aren't squashed into lossy discrete codes. - **Two training objectives at once.** Next-token prediction for discrete sequences, and flow matching for continuous image, video, and action generation, with both streams sharing attention in every block. - **Speed.** The base model generates video at about 0.79x real time (median), with a watchable stream in roughly 6 seconds. A distilled variant cuts denoising from 99 steps to 8 and returned a 5.3-second clip in about a second in Reka's internal tests. - **Modest compute.** The checkpoint was trained on 320 H100 GPUs for about three months. - **Access.** Research preview only. There are no public weights and no public API yet; Reka is inviting collaborators. ## Expected vs. actual **Expected:** to build an app that draws a scene, finds an object in it, animates it, and then explains what changed, you wire together an LLM planner, an image model, a detector, a video model, and a captioner. **Actual:** in Reka's five-turn demo, one model does all of it in one conversation. The bounding box is emitted as coordinate tokens over a scene the model already holds in context. When asked what changed between two videos, it reads the latent state that produced them instead of captioning exported frames. ## A plain way to picture it Think of a relay race where each runner gets a sticky note at the handoff. Every runner only knows their own leg, and every handoff costs time and loses detail. Rho-1 is one runner who has seen the whole course. Nothing is passed along, so nothing gets lost in the pass. ## Three things worth noticing as a builder 1. **Handoffs are a hidden tax.** Each hop in a multi-model pipeline adds latency and strips context, because each specialist sees only a narrow request. Reka's own side-by-side shows its pipeline at 13.8 seconds versus 7.0 seconds for Rho-1 (the pipeline timing is labeled illustrative). 2. **The KV cache becomes a world state.** If generated media stays in context, follow-up edits and questions are grounded in what the model actually made. That is a different design pattern from "generate, export, re-ingest". 3. **Serving gets harder, not easier.** Reka is candid that mixing diffusion steps with autoregressive decoding creates new bottlenecks: discrete KV caches sharing GPU memory with transient latent states, swings between compute-bound and memory-bound phases, and hard real-time latency targets. Standard text-LLM serving stacks weren't built for this. ## The honest limitations Reka lists them itself, which I appreciate: - **Long-horizon drift:** a 30-second stream can keep photoreal texture while the room layout quietly stops making sense. - **Grounding across time:** object detection works on still images, not yet reliably across video. - **Editing stability:** targeted edits are still brittle across prompts. - **Resolution:** native video is capped at 672x384. There are also no standard benchmark tables yet, only internal speed numbers and demos. Treat this as an architectural direction, not a product you can ship on today. ## What I'd do this month - Map your current multimodal or agent pipeline and count the handoffs. Each one is a place where latency and context loss pile up. - Keep generated artifacts (images, structured outputs, intermediate state) in context where you can, instead of round-tripping them through files and re-encoding. - If you work in robotics, simulation, or interactive video, Reka is taking collaborators. For everyone else, watch for weights or an API before betting a roadmap on it. ## References - Reka, "Rho-1: Collapsing the multimodal stack" (Oct 5, 2026): [https://reka.ai/news/rho-1-collapsing-the-multimodal-stack](https://reka.ai/news/rho-1-collapsing-the-multimodal-stack) - The Decoder, "Reka AI's omni-model Rho-1 handles text, images, video, and robot control in a single model" (Oct 5, 2026): [https://the-decoder.com/reka-ais-omni-model-rho-1-handles-text-images-video-and-robot-control-in-a-single-model/](https://the-decoder.com/reka-ais-omni-model-rho-1-handles-text-images-video-and-robot-control-in-a-single-model/) - AlphaSignal, "Rho-1 puts reasoning, video, and robot control in one model" (Oct 2026): [https://alphasignal.ai/news/reka-s-rho-1-merges-reasoning-video-and-robot-control-into-one-model](https://alphasignal.ai/news/reka-s-rho-1-merges-reasoning-video-and-robot-control-into-one-model) - Lipman et al., "Flow Matching for Generative Modeling" (background on the flow-matching objective): [https://arxiv.org/abs/2210.02747](https://arxiv.org/abs/2210.02747) *Speed figures and demos above are Reka's own reported results; no independent benchmarks exist yet.*