Running AV Reasoning Models Within Onboard Compute and Latency Limits

NVIDIA released Alpamayo 2 Edge, a 2B-parameter adaptive reasoning vision-language-action model built on a Nemotron-based backbone and paired with a diffusion action expert, designed to run in real time on NVIDIA DRIVE AGX Thor onboard compute. NVIDIA reports that closed-loop reinforcement learning post-training in AlpaGym improved closed-loop driving performance by 48.9% over the best checkpoint without closed-loop RL, with speculative decoding via DFlash2 and TensorRT quantization recipes used to cut inference cost. The release targets AV developers who need reasoning-based driving models that fit onboard compute and latency budgets.

Autonomous vehicle AV developers need reasoning models that can operate within the compute and latency constraints of an onboard system. Larger models provide a foundation for developing these capabilities, but moving them into a vehicle requires additional work to reduce model size, preserve useful reasoning, and validate performance on the target hardware. NVIDIA Alpamayo 2 Edge https://huggingface.co/nvidia/Alpamayo2-Edge addresses this need with an adaptive reasoning vision-language-action VLA model designed for real-time operation on NVIDIA DRIVE AGX Thor https://developer.nvidia.com/drive/agx/ . Built on the 2B Nemotron-based reasoning backbone of NVIDIA Cosmos 3 Edge https://www.nvidia.com/en-us/ai/cosmos/ and paired with a diffusion action expert, it leverages surround-camera observations, temporal context, and navigation inputs to generate Chain-of-Causation CoC reasoning and future driving trajectories. It reasons adaptively , generating reasoning only when needed, reducing unnecessary token generation and conserving inference compute during routine driving. Alpamayo 2 Edge is post-trained with closed-loop reinforcement learning RL in NVIDIA AlpaGym https://github.com/NVlabs/alpagym , improving closed-loop driving performance by 48.9% on top of the best-performing checkpoint without closed-loop RL. Together, these capabilities give developers a strong, edge-ready foundation for bringing reasoning-based driving into an onboard stack. Making a reasoning model compact enough for onboard deployment introduces a key challenge: preserving the quality of its reasoning and driving behavior as model size is reduced. This requires a way to measure reasoning quality and use that feedback to improve the model. NVIDIA AlpaJudge is the human-calibrated evaluation and reward model used in Alpamayo 2 Edge development to evaluate reasoning traces, select higher-value training data, and provide RL rewards. This post introduces Alpamayo 2 Edge, explains its inference optimization on DRIVE AGX Thor, and explores three ways to use AlpaJudge in AV development workflows: - Run AlpaJudge-RB and AlpaJudge-RF on a PhysicalAI-AV sample, inspect their scores, and adapt the evaluation to policy-generated reasoning - Select training examples that expose useful learning opportunities - Use human-aligned reasoning scores as rewards for post-training Together, these workflows connect an edge-oriented driving model with a repeatable process for measuring and improving it. Bring reasoning-based driving to the edge An onboard driving model must interpret a changing scene and produce a useful trajectory within the system’s inference budget. The key question is how much driving and reasoning capability a model retains under that constraint. To balance model capacity and efficiency, Alpamayo 2 Edge leverages multiple new features: - A 2B-parameter Nemotron-based reasoning backbone, sized for onboard inference - Adaptive reasoning, reducing unnecessary token generation and conserving inference compute during routine driving - Offline distillation, training with CoC reasoning autolabels generated by Alpamayo 2 Super - Closed-loop reinforcement learning post-training in AlpaGym https://github.com/NVlabs/alpagym , in which the model learns from the consequences of its actions end-to-end in simulation - Multi-resolution inputs to efficiently encode surround-view observations online - Speculative decoding with DFlash2 https://github.com/z-lab/dflash to accelerate online reasoning without sacrificing quality - Recipes for quantization and inference acceleration with NVIDIA TensorRT and NVIDIA TensorRT-Edge-LLM Figure 2, below, shows how these choices translate into capability, comparing Alpamayo 2 Edge with Qwen-Drive-1.0 across closed-loop driving, open-loop trajectory prediction, and scene understanding and reasoning. Early partner feedback also highlights Alpamayo 2 Edge’s driving performance and inference speed: “Within days of getting early access to Alpamayo 2 Edge, we had it running closed-loop in NuRec reconstructions of our own London street recordings. In our early simulation tests, Edge kept its lane in more than twice as many runs as Alpamayo 1.5, recovered from lane offsets about 40% more often, and out of the box planned roughly twice as fast as our hand-tuned Alpamayo 1.5 server on the same GPU.” — Metas Simonavicius, CTO, PORT “Alpamayo 2 Edge demonstrated impressive generalization capabilities, reducing displacement error minADE by 13.6% and lateral prediction error RMSE at 3 seconds by 46.1% compared to Alpamayo 2 Super in our zero-shot evaluation, despite substantial differences in sensor configurations. Fine-tuning on our own driving data delivered a further 44.2% reduction in displacement error compared to the Alpamayo 2 Edge baseline. Within just a few days, we went from receiving the model to running our first tests on a real vehicle.” — Luca Crotti, co-founder and motion planning lead, Niulinx “In less than 48 hours, we made NVIDIA’s Alpamayo 2 Edge run at 10 Hz on AGX Thor for real-time planning.”— Hans Salomonsson, CEO and co-founder, Embedl The Alpamayo 2 Edge repository’s inference script https://github.com/NVlabs/alpamayo2-edge/tree/main/src/alpamayo2 edge/inference.py shows how the model operates: It takes in surround-view camera observations over time, navigation text, and historical ego-motion to produce CoC reasoning traces and future trajectories. python import torch import transformers from alpamayo2 edge.checkpoint import load model from alpamayo2 edge.load physical aiavdataset import load physical aiavdataset from alpamayo2 edge.helper import prepare model inputs, to device from alpamayo2 edge.models.utils import fuse traj tokens data = load physical aiavdataset clip model, loading info = load model "nvidia/Alpamayo2-Edge" settings = dict seed=43, top p=0.98, top k=None, temperature=0.6, num traj sets=1, num traj samples=1, max generation length=256, diffusion kwargs={"inference step": 10}, prepared = prepare model inputs data, model.config, model.tokenizer prepared = to device prepared, device="cuda" """Seed text and diffusion sampling, then run the model on prepared inputs.""" torch.manual seed settings "seed" generator = torch.Generator device="cuda" .manual seed settings "seed" with torch.inference mode , torch.autocast device type=torch.device "cuda" .type, dtype=model.dtype, enabled=model.dtype = torch.float32, , : xyz, rot, , extra = model.sample trajectories from data prepared, num traj sets=settings "num traj sets" , num traj samples=settings "num traj samples" , top p=settings "top p" , top k=settings "top k" , temperature=settings "temperature" , max generation length=settings "max generation length" , diffusion kwargs={ settings "diffusion kwargs" , "generator": generator}, return extra=True, In particular, note that the model will now produce special <NOMINAL tokens instead of reasoning traces sometimes, indicating routine driving and conserving inference compute. Retaining quality during deployment When downloading the model from Hugging Face, by default, Alpamayo 2 Edge’s weights are represented in BF16 and it is directly runnable as a PyTorch model on any supported NVIDIA GPU. To run at real-time on DRIVE AGX Thor, however, Alpamayo 2 Edge must be further inference-optimized. The Alpamayo Developer Hub https://github.com/NVlabs/alpamayo-recipes includes the recipe for the DRIVE AGX Thor deployment workflow: stage the Alpamayo 2 Edge checkpoint and DFlash2 drafter, calibrate FP8 scales, export ONNX graphs, build TensorRT engines with TensorRT Edge-LLM plugins, and run the Python harness. The recipe’s reference configuration was measured on DRIVE AGX Thor-X with DRIVE OS 7.2.7.0, CUDA 13.4, and TensorRT 11.3.1.9. After following the Alpamayo 2 Edge deployment recipe to stage the model, evaluation inputs, and engines, run the bundle with python -m alpamayo2edge edgellm.runtime.run \ --engines "$WORK/engines" \ --model "$MODEL" \ --reference "$INPUTS" \ --dflash2 \ --rolling \ --warmup 1 \ --repeat 5 \ --output "$WORK/results.json" The recipe documents export, build, calibration provenance, board requirements, and per-stage measurements so developers can reproduce the latency and paired-quality check on their DRIVE AGX Thor configuration. For reference, the resulting optimized model has a pixel-to-trajectory inference latency of 160 ms to 190 ms depending on the amount of reasoning. The same recipe check compares optimized model outputs with a BF16 PyTorch reference built from identical inputs and seeded action noise. Evaluate reasoning with AlpaJudge Trajectory metrics describe how a model plans or drives. Reasoning evaluation adds another dimension: whether the model’s explanation identifies the relevant scene context and supports the decision it describes. AlpaJudge https://huggingface.co/nvidia/alpajudge provides the human-calibrated critic for this aspect of Alpamayo 2 Edge development as well as prior models . The following workflows describe how to conduct that evaluation and use it for model improvement. Choose the judge for your available inputs AlpaJudge provides two checkpoint variants. Choose the variant according to the evidence available for the example you want to evaluate. | Variant | Required inputs | Returned assessment | | AlpaJudge-RB, reference-based | Question, reference answer, and candidate answer; no images | One alignment probability in 0, 1 | | AlpaJudge-RF, reference-free | Three front-wide and three front-tele frames, ego meta-action history and future, and a candidate rationale | Five rubric probabilities plus action, component, and overall scores | Table 1. AlpaJudge variants use different evidence to assess a candidate rationale RB measures agreement with the supplied reference, rather than independently checking the scene. RF does not require a reference rationale, but it does require synchronized visual evidence and formatted meta-action context. Workflow 1: Run AlpaJudge on a PhysicalAI-AV sample The example notebook https://github.com/NVlabs/alpajudge/blob/main/examples/alpajudge pai.ipynb provides a useful starting point, visualizing the data, meta-action context, candidate rationale, and reference rationale tied to a PhysicalAI-AV sample. The same record can then be rescored with a policy-generated CoC trace. Preserve clip ID, timestamp, camera names, and action context in the output so a low score can be traced back to the scene. In open-loop evaluation, the candidate explanation and trajectory can be compared with recorded annotations. In closed-loop evaluation, the explanation at each policy decision can be scored using the observation and action context from that rollout. A high reasoning score is evidence that a rationale matches the judge’s rubric, while collision, road-departure, close-encounter, and trajectory measures describe different aspects of driving behavior. Report them together. Step 1: Set up the package and open the notebook From the AlpaJudge repository root, install the locked environment, authenticate for the model and PhysicalAI-AV dataset, and open the example notebook: uv sync --locked --dev hf auth login uv run jupyter lab examples/alpajudge pai.ipynb The notebook uses clip 030c760c-ae38-49aa-9ad8-f5650a545d26 at 5.1 s. It decodes front-wide and front-tele frames at −1.0, −0.5, and 0.0 s, oldest first. The example rationale and meta-action strings are manually authored; the dataset supplies the camera frames. Replace all text fields together when using a different moment. Step 2: Download the two judge checkpoints The notebook downloads nvidia/alpajudge once and selects the reference-based/ and reference-free/ subfolders. These are separate checkpoints with separate loaders. Its snapshot call filters to those two folders; set ALPAJUDGE MODEL REVISION to an immutable revision for a reproducible run. A model load needs a CUDA GPU and roughly 5 GB for weights before activations. checkpoint root = snapshot download repo id="nvidia/alpajudge", revision=MODEL REVISION, allow patterns= "reference-based/ ", "reference-free/ " , Step 3: Score the sample and inspect both outputs Run the notebook’s fixed construction-zone sample first. With its recorded checkpoint and inputs, the smoke test reports RB alignment 0.999 and RF overall 0.842. The five RF subscores assess action/reference agreement, action safety, component grounding, component completeness, and causal consistency. The RF model defines action = p match × 1 + p safe /2 and component = mean p grounded, p complete, p causal , then combines them overall = 0.6 × action + 0.4 × component. Core notebook calls: rb model, rb tokenizer = load reference based str checkpoint root , subfolder="reference-based", device="cuda", rb = score reference based rb model, rb tokenizer, { "question": QUESTION, "answer": REFERENCE RATIONALE, "student": CANDIDATE RATIONALE, } , 0 .item rf model, rf processor = load reference free str checkpoint root , subfolder="reference-free", device="cuda", rf = score reference free rf model, rf processor, front wide frames=front wide frames, front tele frames=front tele frames, meta action history=META ACTION HISTORY, meta action future=META ACTION FUTURE, rationale=CANDIDATE RATIONALE, print rb, rf.overall, rf.subscores To evaluate Alpamayo 2 Edge itself, run its inference script on the selected clip and inspect the generated rationale in output.json under text.cot. Use that same clip and decision time when loading AlpaJudge-RF’s six frames, and replace the notebook’s hand-authored rationale with the generated string. Provide meta-action history and future that match this decision rather than copying the notebook’s construction-scene example. If an expert reference CoC trace is available, pass it to RB as answer and the Edge CoC as student . Workflow 2: Select higher-value training examples AlpaJudge’s score is useful for training data selection because the highest and lowest scoring examples often serve different purposes. A very low reference-based score can indicate a mismatched label, a failed rollout, or a genuinely difficult case. Review these cases before discarding them; a blanket removal of low-scoring clips would also remove rare failures the policy needs to learn. Very high scores can identify scenes the current model already explains consistently and is not learning much from anymore. The middle range often contains data that is highly informative for further training. Applying this to your own dataset begins with candidate CoC rollouts per clip and a fixed AlpaJudge checkpoint. Score each candidate against the paired reference if using AlpaJudge-RB , record the score, clip ID, and any scenario metadata, then review low-scoring and safety-critical examples manually. Set selection thresholds on a development split, stratify the retained set by scene type and maneuver, and keep the held-out evaluation split untouched. Resample informative clips for further training and compare the new checkpoint with the same baseline set. Workflow 3: Use reasoning scores for post-training The public Alpamayo RL recipe https://github.com/NVlabs/alpamayo-recipes/blob/main/recipes/alpamayo1 x rl/README.md offers a concrete pattern for adding a reasoning reward to open-loop GRPO training. It generates groups of policy rollouts, decodes each CoC and trajectory, computes trajectory error and comfort, then calls a pluggable reasoning grader before combining the terms into one scalar reward. AlpaJudge-RB can be that grader when each sample has a reference reasoning trace: pass the fixed driving question, reference as answer, and sampled CoC as student, then use the returned alignment probability as the reasoning signal. An adapter can implement the recipe’s BaseReasoningGrader.score predictions, references - Tensor N contract by batching the text pairs through score reference based. Register that adapter in GRADER REGISTRY and point the reasoning reward config to the corresponding checkpoint. The reward function should keep its trajectory gate and log both component scores, so the policy is less likely to earn a high total reward from a fluent but wrong driving explanation. Get started with Alpamayo 2 Edge and AlpaJudge These workflows form a repeatable development loop: run the edge model, score its reasoning next to trajectory and safety metrics, select informative clips, and compare the next candidate against the initial baseline. For teams that also train policies with RL, AlpaJudge can provide a reasoning reward through an adapter such as the one described above. Explore Alpamayo 2 Edge https://huggingface.co/nvidia/Alpamayo2-Edge and AlpaJudge https://huggingface.co/nvidia/alpajudge on Hugging Face. For discussions and developer questions, visit the Alpamayo developer forum https://forums.developer.nvidia.com/c/autonomous-vehicles/alpamayo/766 . Check out the following for more information on: - Alpamayo Open Platform https://www.nvidia.com/en-us/solutions/autonomous-vehicles/alpamayo/ and its latest releases. - State-of-the-art research https://research.nvidia.com/labs/aspire/ informing Alpamayo’s development.