# GridDriver: On-Board BEV-Based Language Planning and Reasoning

> Source: <https://arxiv.org/abs/2611.21924>
> Published: 2026-11-18 05:00:00+00:00

**Recommendation:** For onboard driving LLMs, use spatially structured BEV tokens instead of Q-Former compression and interleave trajectory prediction with asynchronous language generation to hit 32.7 ms planning latency.
More in the playbook "Inference and self-hosting": <https://wpnews.pro/playbooks/inference-serving>

arXiv:2611.21924v1 Announce Type: new 
Abstract: Bird's-Eye-View (BEV)-conditioned large language models (LLMs) enable language-interactive driving, but onboard deployment requires efficient perception and continuous planning alongside autoregressive language generation. We present GridDriver, an efficient BEV-conditioned LLM framework combining trajectory prediction, structured driving indicators, natural-language navigation, driving explanations, and scene question answering. Lightweight spatio-temporal BEV perception and spatially structured tokens provide scene context to a shared LLM, while frame-wise scheduling interleaves trajectory prediction with asynchronous language generation. GridDriver achieves competitive performance on public benchmarks, reaching 87.9% PDMS on NAVSIM and 57.6% accuracy on nuScenes-QA with a shared planning and QA checkpoint. Spatial representation analyses demonstrate improved geometric information retention and trajectory prediction compared with Q-Former compression. The system achieves 32.7 ms standalone planning latency and 9.4 plans/s in an onboard ROS 2 pipeline. We further evaluate planning on our own data recordings and demonstrate open-loop onboard operation on a research vehicle. Across public benchmarks and real-world evaluations, we identify strengths and limitations of the BEV-based approach.
