miniTE: Diagnosing and Bridging the Execution Gap in Native NVFP4 Reinforcement Learning

A study titled "miniTE: Diagnosing and Bridging the Execution Gap in Native NVFP4 Reinforcement Learning" reports that matching model weights and numerical formats is not sufficient to ensure consistent policy evaluation when reinforcement learning for large language models is split across separate training and sampling engines, even with native NVFP4. The work diagnoses this execution gap and proposes a method to bridge it, targeting stable learning in NVFP4-based RL pipelines. The finding matters for teams running disaggregated RL training and inference, where numerical consistency between engines is assumed but not guaranteed.

Native NVFP4 promises to accelerate reinforcement learning RL for large language models LLMs , but achieving stable learning across separate training and sampling engines remains challenging. We show that matching model weights and numerical formats is insufficient to ensure consistent policy eva