A lean visual policy beats a 7B‑parameter VLM
Jeff Cui (@jeffacce) announced on July 22, 2026 a new robot‑control method he calls Patch Policy. The approach freezes a state‑of‑the‑art Vision Transformer (ViT) pretrained on large‑scale image data, feeds every patch token to a lightweight transformer policy, and adds a simple block‑causal mask. The result, Cui writes, “beats OpenVLA‑OFT with 0.7% of its params, and trains on a 5090.”
The claim is backed by an arXiv preprint (arXiv:2607.18236) co‑authored with Gaoyue Zhou (@GaoyueZhou), Ada Langford (@LangfordAd91020), Bowen Tan (@bowentan_), Yann LeCun (@ylecun) and Lerrel Pinto (@LerrelPinto). The paper details experiments on a suite of simulated and real‑world manipulation tasks, comparing three families of visual policies:
Global‑pooled features (e.g., ResNet or ViT CLS token) – the historic baseline. OpenVLA‑OFT, a 7 billion‑parameter vision‑language model fine‑tuned for control. Patch Policy, the new method using dense patch embeddings from a frozen ViT and a small transformer head.
Performance gap
Across the benchmark, Patch Policy improves task success by 40% over global‑pooled policies and by 18% over OpenVLA‑OFT when both are trained on the same tasks. The tiny policy contains only about 0.7 % of the total parameters of OpenVLA‑OFT, yet inference runs at roughly 10 ms latency – six times faster than the VLM baseline – without any hardware‑specific optimization.
Training on consumer hardware
Cui emphasizes that the entire pipeline fits on a single NVIDIA RTX 5090 GPU. “It trains on a 5090,” he notes in the first tweet of the thread. The training cost, therefore, is comparable to that of many academic labs and far below the multi‑GPU clusters typically required for 7 B‑parameter models.
Why dense patches matter
The thread argues that prior robot‑learning pipelines either pooled visual features globally, trained vision models from scratch, or hauled around massive VLMs. By contrast, dense patch features retain spatial detail that is crucial for precise manipulation. A side‑by‑side video in the thread shows a global‑pooled policy failing to locate a 2 mm‑tolerance cable, while the Patch Policy inserts the cable cleanly.
Encoder comparison
Cui’s team benchmarked five frozen encoders as control representations. DINOv2 and WebSSL consistently topped the leaderboard, suggesting that modern self‑supervised vision models provide the most useful dense features for control. The rankings remained stable across tasks and policy heads, reinforcing the importance of the visual backbone.
Open‑source rollout
The authors have opened a project website (patch‑policy.github.io) and a GitHub repo (https://github.com/gaoyuezhou/patch_policy) where code will be released “soon.” Supplemental videos demonstrate the robot inserting a cable, unplugging mid‑rollout, and repeating the maneuver – all with the lightweight policy.
Implications for robot‑learning research
Patch Policy challenges the prevailing belief that scaling model size is the primary driver of better robot control. By showing that off‑the‑shelf dense representations can substitute for massive VLMs, the work may shift investment toward better pretraining of vision backbones and more efficient policy architectures. It also lowers the entry barrier for labs without access to large compute clusters, potentially accelerating experimentation in dense‑perception robotics.
Sources