{"slug": "use-llm-for-direct-robot-control", "title": "Use LLM for direct robot control", "summary": "A paper submitted to arXiv on 20 October 2026 evaluated GPT-6 Astra for closed-loop robotic manipulation and found a pronounced gap between semantic reasoning and physical execution. GPT-6 Astra reached 26.92% success on RoboDojo and 76.0% on RoboTwin 2.0, but only 11.1% zero-shot success on LIBERO-GeoMotion, a new geometry-constrained benchmark the authors introduced requiring precise pose alignment, coordinated translation and rotation, and anticipation of physical action consequences. The authors report that zero-shot execution is feasible for simple tasks, but persistent deficits remain in motion feasibility, precise control and physical safety, with substantial inference latency further hindering practical deployment.", "body_md": "# Computer Science > Robotics\n\n  [Submitted on 20 Oct 2026]\n\n# Title:Large Multimodal Models for Direct Robot Control: GPT-6 as a Beginning\n\n[View PDF](https://arxiv.org/pdf/2610.28259)\n\n[HTML (experimental)](https://arxiv.org/html/2610.28259v1)\n\nAbstract:Recent advances in visuospatial understanding and agentic reasoning enable direct robotic manipulation with general-purpose multimodal models, but their embodied capabilities remain poorly understood. This paper systematically evaluate GPT-6 Astra for closed-loop robotic manipulation using a lightweight agent execution framework. We first probe three fundamental embodied capabilities-depth perception, embodied in-context learning, and tactile understanding-and demonstrate that the model can exploit depth information, acquire simple manipulation strategies from contextual demonstrations, and use tactile feedback to correct its actions. We then evaluate GPT-6 Astra across five established manipulation benchmarks along six capability dimensions. Despite competitive performance on several benchmarks-26.92\\% success on RoboDojo and 76.0\\% on RoboTwin 2.0-our analysis reveals a pronounced gap between semantic reasoning and physical execution, particularly in consequence-aware action planning, precision, and complex motion coordination. To further examine these limitations, we introduce LIBERO-GeoMotion, a new benchmark of geometry-constrained tasks that require precise pose alignment, coordinated translation and rotation, and anticipation of physical action consequences. GPT-6 Astra achieves only 11.1\\% zero-shot success, exposing the difficulty of translating spatial understanding into physically feasible motion. Real-world experiments corroborate these findings: zero-shot execution is feasible for simple tasks, yet persistent deficits remain in motion feasibility, precise control, and physical safety, while substantial inference latency further hinders practical deployment. Overall, our results characterize the emerging embodied capabilities of general-purpose multimodal models and delineate the key challenges toward safe, efficient, and reliable robotic manipulation.\n    \n\n### Additional Features\n\n### References & Citations\n\nLoading...\n\n# Bibliographic and Citation Tools\n\nBibliographic Explorer \n\n*(*[What is the Explorer?](https://info.arxiv.org/labs/showcase.html#arxiv-bibliographic-explorer))\nConnected Papers \n\n*(*[What is Connected Papers?](https://www.connectedpapers.com/about))\nLitmaps \n\n*(*[What is Litmaps?](https://www.litmaps.co/))\nscite Smart Citations \n\n*(*[What are Smart Citations?](https://www.scite.ai/))\n# Code, Data and Media Associated with this Article\n\nalphaXiv \n\n*(*[What is alphaXiv?](https://alphaxiv.org/))\nCatalyzeX Code Finder for Papers \n\n*(*[What is CatalyzeX?](https://www.catalyzex.com))\nDagsHub \n\n*(*[What is DagsHub?](https://dagshub.com/))\nGotit.pub \n\n*(*[What is GotitPub?](http://gotit.pub/faq))\nHugging Face \n\n*(*[What is Huggingface?](https://huggingface.co/huggingface))\nScienceCast \n\n*(*[What is ScienceCast?](https://sciencecast.org/welcome))\n# Demos\n\n# Recommenders and Search Tools\n\nInfluence Flower \n\n*(*[What are Influence Flowers?](https://influencemap.cmlab.dev/))\nCORE Recommender \n\n*(*[What is CORE?](https://core.ac.uk/services/recommender))\n# arXivLabs: experimental projects with community collaborators\n\narXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.\n\nBoth individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.\n\nHave an idea for a project that will add value for arXiv's community? [**Learn more about arXivLabs**](https://info.arxiv.org/labs/index.html).", "url": "https://wpnews.pro/news/use-llm-for-direct-robot-control", "canonical_source": "https://arxiv.org/abs/2610.28259", "published_at": "2026-11-14 17:55:04+00:00", "updated_at": "2026-11-14 18:23:10.606706+00:00", "lang": "en", "topics": ["robotics", "large-language-models", "ai-research", "artificial-intelligence"], "entities": ["GPT-6 Astra", "arXiv", "RoboDojo", "RoboTwin 2.0", "LIBERO-GeoMotion"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/use-llm-for-direct-robot-control", "markdown": "https://wpnews.pro/news/use-llm-for-direct-robot-control.md", "text": "https://wpnews.pro/news/use-llm-for-direct-robot-control.txt", "jsonld": "https://wpnews.pro/news/use-llm-for-direct-robot-control.jsonld"}}