# New robot policy runs on an RTX 5090, beating OpenVLA-OFT while using 0.7% of its size

> Source: <https://runtimewire.com/article/new-robot-policy-runs-on-an-rtx-5090-beating-openvla-oft-while-using-0-7-of-its->
> Published: 2026-07-22 21:48:10+00:00

### A lean visual policy beats a 7B‑parameter VLM

[Jeff Cui (@jeffacce)](https://x.com/jeffacce) announced on July 22, 2026 a new robot‑control method he calls **Patch Policy**. The approach freezes a state‑of‑the‑art Vision Transformer (ViT) pretrained on large‑scale image data, feeds every patch token to a lightweight transformer policy, and adds a simple block‑causal mask. The result, Cui writes, “beats OpenVLA‑OFT with 0.7% of its params, and trains on a 5090.”

The claim is backed by an arXiv preprint (arXiv:2607.18236) co‑authored with [Gaoyue Zhou (@GaoyueZhou)](https://x.com/GaoyueZhou), [Ada Langford (@LangfordAd91020)](https://x.com/LangfordAd91020), [Bowen Tan (@bowentan_)](https://x.com/bowentan_), [Yann LeCun (@ylecun)](https://x.com/ylecun) and [Lerrel Pinto (@LerrelPinto)](https://x.com/LerrelPinto). The paper details experiments on a suite of simulated and real‑world manipulation tasks, comparing three families of visual policies:

**Global‑pooled features** (e.g., ResNet or ViT CLS token) – the historic baseline.
**OpenVLA‑OFT**, a 7 billion‑parameter vision‑language model fine‑tuned for control.
**Patch Policy**, the new method using dense patch embeddings from a frozen ViT and a small transformer head.

#### Performance gap

Across the benchmark, Patch Policy **improves task success by 40% over global‑pooled policies** and **by 18% over OpenVLA‑OFT** when both are trained on the same tasks. The tiny policy contains only about **0.7 % of the total parameters** of OpenVLA‑OFT, yet inference runs at roughly **10 ms latency** – six times faster than the VLM baseline – without any hardware‑specific optimization.

#### Training on consumer hardware

Cui emphasizes that the entire pipeline fits on a single NVIDIA RTX 5090 GPU. “It trains on a 5090,” he notes in the first tweet of the thread. The training cost, therefore, is comparable to that of many academic labs and far below the multi‑GPU clusters typically required for 7 B‑parameter models.

#### Why dense patches matter

The thread argues that prior robot‑learning pipelines either pooled visual features globally, trained vision models from scratch, or hauled around massive VLMs. By contrast, **dense patch features retain spatial detail** that is crucial for precise manipulation. A side‑by‑side video in the thread shows a global‑pooled policy failing to locate a 2 mm‑tolerance cable, while the Patch Policy inserts the cable cleanly.

#### Encoder comparison

Cui’s team benchmarked five frozen encoders as control representations. **DINOv2** and **WebSSL** consistently topped the leaderboard, suggesting that modern self‑supervised vision models provide the most useful dense features for control. The rankings remained stable across tasks and policy heads, reinforcing the importance of the visual backbone.

#### Open‑source rollout

The authors have opened a project website ([patch‑policy.github.io](https://patch-policy.github.io/)) and a GitHub repo ([https://github.com/gaoyuezhou/patch_policy](https://github.com/gaoyuezhou/patch_policy)) where code will be released “soon.” Supplemental videos demonstrate the robot inserting a cable, unplugging mid‑rollout, and repeating the maneuver – all with the lightweight policy.

#### Implications for robot‑learning research

Patch Policy challenges the prevailing belief that scaling model size is the primary driver of better robot control. By showing that **off‑the‑shelf dense representations can substitute for massive VLMs**, the work may shift investment toward better pretraining of vision backbones and more efficient policy architectures. It also lowers the entry barrier for labs without access to large compute clusters, potentially accelerating experimentation in dense‑perception robotics.

**Sources**