{"slug": "griddriver-on-board-bev-based-language-planning-and-reasoning", "title": "GridDriver: On-Board BEV-Based Language Planning and Reasoning", "summary": "GridDriver, a BEV-conditioned large language model framework for onboard driving, reached 87.9% PDMS on the NAVSIM benchmark and 57.6% accuracy on nuScenes-QA using a single shared planning and question-answering checkpoint, according to the arXiv paper 2611.21924v1. The system combines lightweight spatio-temporal BEV perception with spatially structured tokens and frame-wise scheduling that interleaves trajectory prediction with asynchronous language generation, achieving 32.7 ms standalone planning latency and 9.4 plans per second in an onboard ROS 2 pipeline. The authors report open-loop onboard operation on a research vehicle and say spatial representation analyses show better geometric information retention than Q-Former compression.", "body_md": "arXiv:2611.21924v1 Announce Type: new \nAbstract: Bird's-Eye-View (BEV)-conditioned large language models (LLMs) enable language-interactive driving, but onboard deployment requires efficient perception and continuous planning alongside autoregressive language generation. We present GridDriver, an efficient BEV-conditioned LLM framework combining trajectory prediction, structured driving indicators, natural-language navigation, driving explanations, and scene question answering. Lightweight spatio-temporal BEV perception and spatially structured tokens provide scene context to a shared LLM, while frame-wise scheduling interleaves trajectory prediction with asynchronous language generation. GridDriver achieves competitive performance on public benchmarks, reaching 87.9% PDMS on NAVSIM and 57.6% accuracy on nuScenes-QA with a shared planning and QA checkpoint. Spatial representation analyses demonstrate improved geometric information retention and trajectory prediction compared with Q-Former compression. The system achieves 32.7 ms standalone planning latency and 9.4 plans/s in an onboard ROS 2 pipeline. We further evaluate planning on our own data recordings and demonstrate open-loop onboard operation on a research vehicle. Across public benchmarks and real-world evaluations, we identify strengths and limitations of the BEV-based approach.", "url": "https://wpnews.pro/news/griddriver-on-board-bev-based-language-planning-and-reasoning", "canonical_source": "https://arxiv.org/abs/2611.21924", "published_at": "2026-11-18 05:00:00+00:00", "updated_at": "2026-11-18 05:15:21.247803+00:00", "lang": "en", "topics": ["autonomous-vehicles", "large-language-models", "computer-vision", "ai-research"], "entities": ["GridDriver", "NAVSIM", "nuScenes-QA", "ROS 2", "Q-Former", "arXiv"], "also_reported_by": [], "takeaway": "GridDriver's spatially structured BEV tokens retain more geometric information than Q-Former compression, reaching 87.9% PDMS on NAVSIM at 32.7 ms planning latency and 9.4 plans/s in an onboard ROS 2 pipeline.", "recommendation": {"advice": "For onboard driving LLMs, use spatially structured BEV tokens instead of Q-Former compression and interleave trajectory prediction with asynchronous language generation to hit 32.7 ms planning latency.", "playbook": "inference-serving", "playbook_url": "https://wpnews.pro/playbooks/inference-serving"}, "alternates": {"html": "https://wpnews.pro/news/griddriver-on-board-bev-based-language-planning-and-reasoning", "markdown": "https://wpnews.pro/news/griddriver-on-board-bev-based-language-planning-and-reasoning.md", "text": "https://wpnews.pro/news/griddriver-on-board-bev-based-language-planning-and-reasoning.txt", "jsonld": "https://wpnews.pro/news/griddriver-on-board-bev-based-language-planning-and-reasoning.jsonld"}}