Why Robot Brains Will Live in the Cloud

Physical Intelligence and other robotics researchers argue that future generalist robot 'brains' will run in the cloud rather than on-board, because scaling laws demand models too large for robot GPUs. The company cites evidence from DeepMind's RT-2 and its own π0 series, noting that models smaller than 7 billion parameters fail to generalize, while larger models show promise. This shift reframes the problem from compressing intelligence into small models to engineering reliable, low-latency remote inference.

The promise of generalist robotics is automating physical work such that no human has to perform dirty, dangerous, or demeaning labor. To achieve this, robots must be able to perform a wide range of tasks, adapt to a broad variety of environments, and handle an indefinite number of unexpected scenarios. This requires much more intelligent physical AI models than we have now, especially ones that are trained on extremely large training corpora and are equipped with deep reasoning capabilities. We think this necessarily results in a substantially larger model that just won’t be able to fit on GPUs on-board the robot, and that these ‘brains’ will live in the cloud where they can be served on powerful servers. Conventional wisdom states that running robotic AI models in the cloud is a terrible idea. For robots, a fast reaction speed is critical for both performance and safety, and running the model over the internet is slow and unreliable. Robot builders want to run the most intelligent models which tend to be large, but due to this seemingly inevitable challenge of remote inference, they are forced to work with the constraints of running smaller models on board the robot itself. Furthermore, how can contending with network latency possibly allow robots to react quickly and safely to spontaneous changes in the world? It appears that off-board inference introduces an insurmountable number of engineering problems relating to speed, reliability, control and safety, such that it seems much more straightforward to focus on optimising on-device infrastructure. After all, running models locally guarantees the tight control loops that enable smooth and responsive actuator reactions to dynamic environments — which is the reason why a lot of recent research efforts have gone into making inference efficient on edge compute. However, we believe that we can increase the tractability of this ‘fast intelligence’ problem by flipping the direction of thinking on its head — taking it from the realm of pursuing difficult research that compresses intelligence into a small number of parameters, to the realm of solving the engineering challenges to enable large, capable models to run quickly and reliably over a remote connection. In this post, we’ll walk through the limitations of on-board inference, how cloud inference addresses these problems, then ultimately show how, with the right tradeoffs, a future for off-board inference can and should exist. Scaling Laws are Here to Stay We, like others https://youtu.be/3Y8aq ofEVs?t=753 , also believe that much of the next waves of progress for generalist robotics R&D will adhere to scaling laws. We first saw sparks of general world knowledge in embodied AI models with DeepMind’s RT-2. Its demo of moving a coke can to an image of Taylor Swift was only possible by leveraging a VLM’s web-scale pre-training to understand information about the world. Robot specific data has scaled with π0, and continues strongly with models up to π0.7 and beyond , yielding transfer to unseen environments and objects. Generalist AI uses half a million hours of robot data to achieve exceptionally robust dextrous manipulation. They’ve provided generous evidence for the existence of scaling laws, analogous to LLMs, for embodied AI. Beyond just data, they've also reported that robot performance increases with model size. Models smaller than 7 billion parameters ossify and are unable to learn beyond a certain point, while models in the tens of billions have shown more promise of generalizing. This need for larger model size is accelerated by the recent paradigm shift towards world models , leveraging future state prediction and spatiotemporal priors from video foundation models. Physical Intelligence has also hinted https://www.youtube.com/watch?v=pzolgvyWEFY at moving up one order of magnitude for parameter count too, scaling even to high double-digit billions. While they haven’t quite gotten there yet, we see early evidence with their latest π0.7 VLA’s use of a 14B world model for subgoal image prediction. For companies like NVIDIA and 1X who use World Action Models WAMs , these models are even more compute-intensive. NVIDIA’s DreamZero 14B model takes 5.7 seconds to generate a single action chunk even on a H100 GPU. It requires two of the powerful, server-grade GB200 instances with a series of complex system-level and implementation-level optimisations to run a Flash variant at the advertised 7 hz closed-loop control. And while 1X doesn’t disclose information on their models extensively, we know it takes 11 seconds for their 14B world model to generate a 5 second future video prediction. It’s clear that scaling laws are here to stay. The State of On-board Compute Right now, there are only a handful of hardware options for onboard inference. Many frontier robotics labs today, including 1X and Boston Dynamics, rely on NVIDIA’s Jetson AGX series for running AI models. Let’s take a closer look at two of the most popular offerings in this series: | Performance | RAM | Memory bandwidth | | |---|---|---|---| Jetson AGX Orin | 275 TOPS | 64 GB | 200 GB/s | Jetson AGX Thor | 2070 TFLOPS | 128 GB | 273 GB/s | Both the Orin and the Thor have a healthy amount of RAM, enough to fit any SOTA physical AI model and comfortably run it in theory. But in practice, this isn’t the case — even if you could fit the model, the inference latency is punishing. To understand this, let’s explore exactly how physical AI models push the boundaries of GPU utilization at the architectural level. In a transformer, processing tokens “prefill” is compute bound, and autoregressive generation is memory bandwidth bound due to the KV cache operations constantly moving cache between VRAM and GPU compute cores. On the other hand, the diffusion is bounded by FLOPS, as it involves many repetitive calculations. Usually, VLAs and WAMs rely on both autoregressive and diffusion operations. VLAs are based on multimodal VLMs and attach a smaller diffusion action expert onto them. Typically, the VLM is used for encoding intermediate representations in its KV cache, which the action expert uses to generate actions. As this is a prefill operation and not generative, we are compute bound. However, similar to LLMs, VLAs e.g. π0.5 are able to reason through difficult tasks with test time compute. When you input a broad prompt like “ clean the bedroom ”, the VLM employs subtask prediction to break this long-horizon task into smaller, more manageable and well-specified sequential steps that allow the model to execute a task with greater success. While the sequences generated are small, you can extrapolate that as models acquire more complex reasoning capabilities, KV cache utilisation and memory bandwidth will become strained. There is nuance in the action expert however. Although a typical diffusion model is compute-bound, inside VLAs, it is actually memory-bound https://arxiv.org/pdf/2602.18397 . This is due to the relatively small number and size of action tokens compared to the KV cache it borrows from the VLM. On the other hand, WAMs autoregressively generate frames; as they are generating future predictions frame by frame, they re-use their KV cache often. But even worse, each generation of each individual frame undergoes diffusion. Both of these processes are extremely intensive — videos are heavy to work with. Therefore, it’s clear that we face a two-pronged problem with both memory and compute constraints if we are running a Thor or Orin on the robot. We know chips will continue to get better year after year. However, we think the demand for stronger intelligence and hence necessarily model size is going to far outpace Huang’s Law in the near future. The converse cannot be true because of Jevon’s paradox; the advances in energy efficiency will only lead to an increased consumption of compute. Hence, it is inevitable that model size will reach a certain ‘escape velocity’ at which it will be out of reach for on-board inference and only cloud inference will be feasible. Indeed, PI’s recent partnership with Modal https://modal.com/blog/physical-intelligence-runs-real-time-remote-inference-for-robotic-control-on-modal offers an interesting case study of how real-time robot execution can be achieved using a model served from a remote data center. In the absence of technical details, one aspect worth noting is that the latency measurements were performed in optimal conditions, over a tightly controlled network, within a specific region close to their servers, solely for research purposes. Regardless, it offers a glimpse into the possibilities into a cloud future. Irrespective of how you look at it, there’s a bitter lesson for robotics to be found here. Models consistently get better when you make them bigger and give them more data. We believe that off-board inference is the necessary infrastructure to support this future. Impacts on Robot Cost, Hardware, Access, and Beyond Cost From a logistical perspective, one of the biggest problems of putting a GPU inside the robot is that it is extremely expensive and has become difficult to acquire at scale. It is clear that cloud inference replaces this upfront cost in the form of usage-based pricing. But how does this ongoing cost of cloud inference pan out against the one-time cost of purchasing a GPU over a long time horizon? Let’s do some napkin math. Suppose we have a robot equipped with an NVIDIA AGX Thor, costing $3500. NVIDIA’s Thor has a rated lifespan of 5 years https://developer.download.nvidia.com/assets/embedded/secure/jetson/thor/docs/Jetson Thor Series Modules Datasheet DS-11945-001v1.5.pdf?t=eyJscyI6ImdzZW8iLCJsc2QiOiJodHRwczovL3d3dy5nb29nbGUuY29tLyJ9& token =exp=1783870838~hmac=0adcc9cc342664b0325cdd22d46a1932d40db5061e242c4fad0d108a7156a78d , which includes testing with heat-cooling cycles and vibrations. This equates to a lifetime amortised cost of $0.08/hour. Meanwhile, an H100 instance on AWS currently costs $6.88/hour, which is 86x more expensive. While this appears to be a large gap, the more relevant measure is the cost per hour of productive work. In practice, most robots won’t operate 24/7/365: they require maintenance, run out of battery, wait between tasks and sit idle when demand is low. These periods reduce the number of useful hours over which the upfront hardware cost can be amortised. By paying for compute only when the robot is actually generating economic value and leaving someone else to absorb the cost of idle capacity and hardware depreciation, and multiplexing the execution of multiple robots on a single data center GPU, off-board compute may ultimately be cheaper in the long run. Furthermore, by analogy, renting a house will always cost more in the long run than buying it outright. The landlord bears the upfront cost and, in return, gives tenants access to a valuable resource without the prohibitive barrier to entry. That access alone has value: it unlocks new capabilities for tenants in the housing example, and for researchers and tinkerers in ours. In fact, we would contend that the jaw-dropping claim that a data center GPU “ costs 86x more ” over the long run presents a strawmanned and incomplete picture for one underappreciated reason: H100s are not merely more expensive, they are appreciably more powerful. Imagine, as a thought experiment, that the wizards at NVIDIA produced an H100 with the power draw and form factor of a Thor. Something we could simply slap onboard. At current market rates, this magical H100 would go for around $35,000, which under the same amortisation math comes to roughly $0.80 per hour. If you only had to pay 10× more but gained a 100× uplift in intelligence, would you call that “more expensive”... or simply a better deal? Power & Hardware One important physical consideration is the impact of GPU’s power draw on the robot’s battery. Consider a Unitree H1 humanoid robot, with a reported battery size of 864Wh and a battery life of approximately 2 hours, which gives an implied power draw of the robot alone being 432W. A Jetson Thor can draw 120W, meaning the power draw with the GPU added is 552W, giving a battery life of 1.56 hours. This means that running a powerful GPU on the robot can cause it to lose ~20% of battery life on top of increased battery wear . This means you either need a more performant battery, which in turn makes the robot heavier and the actuators draw more power, diminishing the returns of added charge, or accept reduced operating hours and hot-swap the batteries more often. Furthermore, having a heat-generating GPU on board means the robot now has to be equipped with a cooling system, the chassis must be designed to accommodate them, and places restrictions on material choice e.g. can’t use thermoplastics . Running GPU inference on a cloud data center with optimized cooling and abundant electricity supply means lighter robots and less hardware components and restrictions to worry about. Access A big problem for tinkerers in Physical AI is that access to running performant models on their robots is gated by owning a powerful GPU. Small labs and development teams lack the capital to purchase compute upfront, let alone have the time to wait for it to arrive. GPU access is load-bearing to the R&D cycle. Waiting for one of your 5 GPUs to free up to test a policy kills development velocity. But this isn’t exclusive to small, scrappy teams - Physical Intelligence themselves say " running compute on cloud simplified operations and enabled rapid experimentation with larger models, while only adding 10-15ms of network overhead ". As we can see, off-board inference offers a significant operational advantage on the ease and velocity of development. Safety Larger, more capable models are safer robots. In some sense, safety is intrinsically tied to capability. It’s extremely likely that larger and more diversely trained robot models will generalise better to unseen, complex domains, as well as attain stronger perception and semantic understanding. For instance, more complex emergencies e.g. rescuing people during house fires will require more complex reasoning patterns to resolve, so it’s reasonable to infer that the most capable models should be deployed in these high-stakes situations. Large models will be generally better at recognising hazards, reasoning through unfamiliar situations and instruction-following for safety policies. Bigger models will reduce embodied mistakes arising from weak perception and reasoning. It’s also generally dangerous for robots to have slow reflexes. Every millisecond counts. We don’t want embodiments to wait 300 ms after they’ve crashed into a child to turn in a different direction, nor pull heavy machinery back after it has collided with a pillar. The combination of heavy equipment and strong forces causes serious harm. But accidents can be mitigated if your robot has good reflexes in the same way humans do, so the current paradigm has been to strap on a GPU on-device. It’s therefore simply counter-intuitive to add additional overhead from the network, preventing robots from reacting in time. The latest generations of π models run at 50 Hz. But note that the model isn’t actually doing 50 forward passes per second. This level of responsiveness is achieved via action chunking, a technique which involves generating a set of 50 future actions each forward pass, then executing them over 1 second. It actually runs inference 3 times per second — at 3 Hz, which is terrible for safety; it’s a woefully inadequate response rate. Thinking, Fast and Slow will help us here. We can emulate Kahneman’s distinct modes of thought by decoupling a dedicated, on-device System 1 from our large System 2 robot model that lives in the cloud. To illustrate this with an example, Figure AI’s Helix 02’s System 1 actually called System 0, since they run 3 distinct systems , is a tiny 10M model running locally whose sole task is to keep the robot coordinated and balanced. This foundation model operates at 1 kHz, and takes in joint states and base motion as input, so it can reliably output actuator commands in time for safety. The systems above it can relax a little. Helix 02 is just an example, but we think that this could serve as inspiration for a very real and optimal shape for robot control going forward: a hybrid system where body control and safety is handled by a lightweight fallback model/policy that feasibly sits on-device. The rest is put off-board to leverage the highest performance possible for dexterity, reasoning and conversation. Critiques on Off-board Inference Here, we address potential concerns against off-board inference. 1. How do you deal with the slow and unpredictable latency of the network? While this is a known limitation of all cloud-based systems, we firmly believe that the stochasticity of network latency is a tractable engineering problem. Consider that in 2024, doctors performed telesurgery over 8000 kilometers from Rome to Beijing with a controlled delay of only 135 ms. They used hedging across both a high-speed 5G network and fiber-optic connection. By leveraging network transport optimizations like QUIC Quick UDP Internet Connections and zero-copy data transfer, we can bring down the raw round-trip-time and jitter between a cloud inference endpoint and an optimised networking stack running on a lightweight embedded system inside the robot. We are also excited about the latest WiFi 7 and the upcoming WiFi 8 protocols, which enable theoretical speeds up to 46 Gbps and are designed specifically for ultra-high reliability. Furthermore, unlike traditional LLM inference providers which allocate large amounts of compute in concentrated regions to serve broad regions, we believe that latency-sensitive robots will be served by compact clusters of hyper-local compute targeting specific subregions. Combined with the transport-level network optimizations, we believe that we can bring the network overhead problem into the scope of a manageable issue. 2. What if the robot’s operating in an environment that doesn’t have network connectivity? We recognise that not all corners of the Earth are reachable over the network, such as deep underwater, inside a mine, or on an off-grid farmland. However, we are encouraged by the growing coverage of the Starlink satellite network, offering 25-60 ms of latency on land and continuing to become faster. For latency-critical scenarios, we potentially envision a future of inference via on-premises private cloud, in the form of server racks physically delivered to inference locations. In fact, Anduril has already started deploying container-sized data centers https://www.anduril.com/news/menace-i-brings-petabyte-scale-processing-to-the-warfighter-at-the-tactical-edge%20 to areas where military operations are being conducted, delivering reliable and high-speed processing power to remote locations. 3. Won’t models become compact and efficient enough to fit on an onboard GPU? There is an argument that we will develop models with a high intelligence to parameter count ratio such that sufficiently capable models will be able to fit on onboard GPUs. However, model optimizations are essentially a tradeoff of compactness and accuracy. Methods like quantisation and distillation degrade performance after a certain threshold, and once the base models themselves grow very large, their compressed variants will too. We do recognize that there are, and will continue to be, genuine improvements in model efficiency. One example is Fast-WAM, a world action model which achieves 190 ms latency by removing video prediction at test-time and instead showing that they can attain comparable performance by leveraging the spatiotemporal priors from the pretraining step only. However, we don’t believe there will be a “ sufficiently capable ” model for physical intelligence that fits comfortably on an onboard GPU for a long time. Just like how despite the existence of many efficient open-source models, the world still yearns for even higher levels of capability for AI e.g. GPT 5.6 Sol, Claude Mythos to perform even more meaningfully impactful and economically productive tasks. We are far from, if ever, a future in which truly generalist models are able to be contained in a single GPU. Future Ultimately, we think the future of generalist robotics is dependent on off-board compute. With the ability to run large-scale models at highly optimised latency and reliability, we will see an exponential uplift in the capability and safety of general-purpose robots. The landscape of robotics is already changing. It’s not just factories that get to use robots. It’s the farmers off the grid in rural Australia, it’s the laundry shop down the street, it’s the apartments that need cleaning across NYC. Turning towards a longer time horizon, we could plausibly see hyper-local mini-datacenters being deployed across the world wherever fleets of robots are at work - from neighborhoods to mines to hospitals. There could even be mobile GPU compute boxes and dedicated network access points that serve clusters of nearby robots in every corner of the world. Eventually, we envision a future of intelligent and safe robots, each conquering physical problems in versatile environments, enabled by the nexus of powerful cloud compute and optimised network transport. This is the future we want to engineer into existence.