Black Forest Labs opens FLUX 3 Action, a 7B model that predicts robot moves and video frames together

Black Forest Labs released FLUX 3 Action, an open-weight 7B world-action model derived from its multimodal FLUX 3 backbone, which sets a new state-of-the-art success rate on the RoboLab-120 leaderboard at less than half the parameters of the previous best open model while running up to 3.95x faster. The company reports that combining fast action prediction with frontier reasoning models yields up to 53.64% more success per dollar, and that on a B200 GPU the prior best open world-action model, Cosmos 3 Nano in FP8, needs about 4.7x as much processing time per second of robot motion as the VLA model Pi0.5 in BF16. Black Forest Labs published the weights alongside a report detailing pretraining, midtraining, finetuning and inference optimizations for local deployment.

Table of Contents FLUX 3 Action Visual Intelligence describes systems that understand, reason, perceive and act in the visual world - for example by simulating environments, modifying and editing pixels, or by predicting how to interact with an environment to reach a goal. The latter capability is often referred to as “action prediction” - the ability to predict the next action of an agent and the resulting state of its environment from a series of observations, such as videos from multiple cameras on a robotic workstation. The advance of multimodal world models has enabled rapid progress for systems that interact with the world, especially in the field of robotics - while at the same time, frontier reasoning systems have shown strong planning and generalization capabilities but remain practically limited by latency and model size. We present FLUX 3 Action, an open-weight 7B world-action-model derived from our multimodal FLUX 3 backbone, that was pretrained on a large-scale collection of video, image and audio data - with a strong emphasis on videos https://bfl.ai/blog/flux-3 . On the RoboLab-120 https://research.nvidia.com/labs/srl/projects/robolab/leaderboard.html leaderboard it sets a new state-of-the-art success rate at less than half the parameters of the previous best open model, while running up to 3.95x faster. Images, video and audio represent different aspects of the underlying reality. Training across these modalities lets us build on a much broader source of data than action demonstrations alone, resulting in more generalization. We then adapt this foundation through joint video-action training and finetuning for a target embodiment and its corresponding action space. In robotics, the fine-tuning recipe is just as important as the weights. We are publishing this report along with the weights to make this process transparent; from the pretraining and midtraining phases to finetuning and inference optimizations for action prediction - where efficiency in particular is a critical and necessary capability for local deployments. In addition, we analyze hybrid systems that combine fast action prediction with the planning capabilities of frontier reasoning models - and find that fast control makes embodied reasoning more cost- and time efficient - up to 53.64% more success per dollar. The focus of this report is action prediction applied to robotics - but action prediction extends to digital environments as well. We explore gaming as a testbed for navigation, anticipating computer-use and other latency-sensitive agents and planning capabilities driven by visual information. Action Policies Open action policies currently force a choice on you: World Action Models WAMs , which predict future actions jointly with video, lead benchmarks such as RoboLab https://research.nvidia.com/labs/srl/projects/robolab/leaderboard.html <sup 1 R 15dbsnpfcpflb -note-0 </sup . Cosmos 3 Nano<sup 2 R 15dbsnpfcpflb -note-1 </sup succeeds on 36.8% of tasks, while Pi0.5<sup 3 R 15dbsnpfcpflb -note-2 </sup , the strongest open Vision Language Action VLA model succeeds on 28.0%. But predicting videos along with actions makes WAMs expensive. On a B200 GPU, Cosmos 3 Nano in FP8 requires about 4.7x as much processing time per second of robot motion as Pi0.5 in BF16. When you choose a WAM you pay in latency and hardware costs; when you choose a VLA you give up on almost a quarter of successful rollouts. Running predictions jointly over video and actions allow WAMs to transfer their world understanding - obtained through large scale video pretraining - to action prediction with limited additional action data supervision. But it also results in long sequence lengths that increase their computational costs. In addition, they often rely on guidance techniques doubling the cost of every forward pass, and run on backbones with more parameters, such as Cosmos 3 having 4.85x more parameters compared to Pi0.5. Existing solutions attempt to limit the interaction between video and actions but in doing so fundamentally change the architecture and capabilities of WAMs. Other approaches rely on parallelization of guidance but lead to increased hardware costs and even then do not reach the same speed as VLAs due to the computational footprint of their larger architectures. FLUX 3 Action F3A simplifies this choice. Our single step 7B checkpoint outperforms every other open policy on RoboLab at 38.3% ± 0.38 compared to 36.8% for Cosmos 3 Nano, while featuring a 1.34x through 2.28x improvement in speed per second of robot motion compared to Pi0.5 on workstation and datacenter GPUs, despite still predicting video and actions jointly. Note that this speed advantage is in real-time factor, not latency per call: As a WAM, it can predict a longer horizon of 2.13s of motion compared to 1.0s for Pi0.5. When latency matters less, our guidance-distilled checkpoint raises the state-of-the-art success rate on RoboLab to 42.2% ± 0.36. Our base and guidance-distilled checkpoints in FP8 provide a 1.52x through 3.95x speedup over Cosmos 3 Nano in FP8 across consumer, workstation and datacenter GPUs. | Model | Closed source | Type | SR% | Parameters | |---|---|---|---|---| | FLUX 3 Action | No | WAM | 42.92% | 7B | | OASIS WAM | Yes | VLM + WAM | 39.0% | – | | Cosmos3-Nano-Policy | No | WAM | 36.8% | 16B | | Phoenix | Yes | TAMP+FM | 34.4% | – | | BiMind v0.1 | Yes | VLA | 33.3% | – | | π0.5 | No | VLA | 28.0% | 3.3B | | DreamZero | No | WAM | 25.7% | 14B | | Cosmos3-Edge-Policy | No | WAM | 22.9% | 4B | | π0-FAST | No | VLA | 15.5% | 3B | | GR00T N1.6 | No | VLA | 7.2% | 3B | | π0 | No | VLA | 5.0% | 3.3B | | paligemma-binning | No | VLA | 3.4% | 3B | RoboLab leaderboard https://research.nvidia.com/labs/srl/projects/robolab/leaderboard.html . Two main changes led to these results: First, a backbone less than half the size of Cosmos 3 Nano, made possible through our multimodal Self-Flow<sup 4 R 45dbsnpfcpflb -note-0 </sup pretraining which yields strong representations. Second, distillation that removes the need for a guidance pass and reduces the number of sampling steps down to a single one, without separating video from action. While this moves the Pareto frontier of action policies across speed and accuracy, there are still tasks that all policies, ours included, fail to solve on their own. GPT 6 Astra on the other hand can solve all tasks in the benchmark of Su et al. 2026 <sup 5 R 45dbsnpfcpflb -note-1 </sup when using maximum reasoning efforts. But a single reasoning call adds around 35.77 seconds per second of robot motion, and each success costs $13.47 and 16 minutes on average. In contrast, F3A on its own takes 21.08 milliseconds per second of robot motion and each success costs $0.09 and less than two minutes on average when running on a H200 GPU for $3 per hour. For the most part of those two minutes, F3A is sitting idle, waiting while the robot moves. In this setting, F3A could even serve 47 other rollouts at the same time without additional costs. The main caveat being that it can’t solve every individual tasks. A promising fix is to let a reasoner, GPT 6 Astra, delegate control to an efficient policy such as F3A or Pi0.5 and only step in when needed. In the setting of Su et al. 2026 , such a hybrid policy barely improves the situation when using Pi0.5. The best hybrid costs $12.28 per success against $13.47 for pure reasoning, and takes 13.5 minutes instead of 14. With F3A as the policy however, this delegation strategy pays off. The hybrid policy still solves 90% of all episodes, including the complex tasks no pure action policy can solve on its own, and it reduces costs to $8.77 and 8 minutes per success, making it 29% cheaper and 40% faster than the best alternative configuration. Pure reasoning with maximum effort still reaches the highest accuracy, so the trade-off between reliability and cost remains, but improved policies such as F3A change where this trade-off sits. The better the fast policy is, the less the system has to reason. These explorations are only a first look into deeper integrations of reasoning and fast action prediction systems, and we expect that there is a lot of room for improvement towards using as much reasoning as necessary and as little as possible. In the following, we describe the development of FLUX 3 Action and our findings along the way. From Video Prediction to Action Prediction The idea to use predictive models of future observations for decision making has a long history<sup 1 R 17dbsnpfcpflb -note-0 </sup , including early demonstrations of robot control<sup 2 R 17dbsnpfcpflb -note-1 </sup which combined action-conditioned video prediction with model-predictive control. UniPi instead<sup 3 R 17dbsnpfcpflb -note-2 </sup generates a video of the task being performed with a text-conditioned video generator and recovers executable actions from the generated frames with an inverse dynamics model IDM . A key argument of UniPi was that pretraining the video model on large-scale non-robotic data transfers to the robot policy and improves generalization to unseen tasks. Subsequent work retained this use of large-scale video pretraining but coupled video and action prediction more tightly than the two-stage pipeline of video generation followed by an IDM. Joint prediction of future frames and actions within a single generative model was explored by GR-1, GR-2, WorldVLA, Cosmos Policy and LingBot-VA<sup 4 R 17dbsnpfcpflb -note-3 </sup ; DreamZero introduced<sup 5 R 17dbsnpfcpflb -note-4 </sup the term World Action Model WAM for this approach, which subsequent systems such as GigaWorld-Policy and the policy variants of Cosmos 3 also adopt<sup 6 R 17dbsnpfcpflb -note-5 </sup . mimic-video instead retains an inverse dynamics model but conditions it on the video model’s latent features rather than decoded frames, avoiding full video synthesis at inference and yielding what the authors call a Video-Action Model VAM <sup 7 R 17dbsnpfcpflb -note-6 </sup . A further line treats video prediction as an auxiliary co-training task and skips video generation at inference altogether<sup 8 R 17dbsnpfcpflb -note-7 </sup . Previously, we explored the performance of FLUX 3 as a VAM with separate action decoder in our FLUX-mimic design study https://bfl.ai/blog/flux-3-mimic . See also mimic’s continued study on the data efficiency of flux-mimic https://www.mimicrobotics.com/blog/video-action-models-for-data-efficient-industrial-manipulation . Here, we study the performance of FLUX 3 as a WAM with joint video-action prediction as shown in Figure 3. Try the Control Loop Given an instruction such as “put the red cube in the left bin,” the model uses camera images and joint positions to predict motor commands and their visual outcomes together. The robot executes a set number of those actions, then looks again and plans from the fresh observation. Start a mission, drop a cube, and watch the arm recover. Change how often it looks again to see how that affects its response. Findings Pretraining and Midtraining Pretraining followed FLUX 3 and used a mix of image, video and audio data, with video accounting for over 95% of the training tokens. To evaluate action prediction capabilities of pretraining checkpoints, we follow a protocol similar to Cosmos 3: we add randomly initialized action heads and then finetune on DROID. The resulting checkpoint is evaluated on RoboLab. For the results in Figure 4, we used preliminary settings with a small batch size and learning rate. As such, they do not represent the maximum performance achievable by a checkpoint. But since all runs used comparable settings, it still allows us to compare their relative capabilities for action prediction. Note that without pretraining, performance remains below 1%. With pretraining, this jumps to 11.6% and continues to improve slowly but steadily to 12.4%. The low performance of the non-pretrained checkpoint indicates that pure DROID action training is heavily data constrained, and it shows that video-heavy multimodal pretraining can indeed overcome such limitation. Next, we consider an action focused midtraining phase, where we continue training on a mix of our pretraining data together with newly introduced action data. We allocate 36.95% of training samples to videos with audio from our pretraining data and the remaining 63.05% to video data that comes with aligned actions. Of all training samples, 19.55% come from game recordings, 13.54% from egocentric videos with hand pose annotations, 14.03% from handheld grippers and 15.93% from teleoperation across 14 embodiments. The corresponding token ratios differ slightly due to different resolutions and durations; see Table 2. | Category | Sample ratio % | Token ratio % | |---|---|---| | Joint video/audio | 36.95 | 39.00 | | Gaming | 19.55 | 21.85 | | Egocentric human hands | 13.54 | 13.78 | | Handheld | 14.03 | 13.99 | | Teleop | 15.93 | 11.37 | Gaming actions are represented with 64 dimensions. The first two encode mouse movement along the x- and y-axes, the next two encode the left and right mouse buttons, and the remaining 60 encode keyboard buttons. The binary button channels are dequantized during training. Most pose-based egocentric, handheld and teleop datasets use a 50 dimensional action space, EE50. The first 25 dimensions are used for the left or primary end-effector and the last 25 for the right or secondary end-effector. Within these 25 dimensions, the first three encode translation, and the next six encode rotation as in Zhou et al. 2019<sup 1 R 19bdbsnpfcpflb -note-0 </sup , but using the first two rows instead of the first two columns of the rotation matrix. The remaining 16 dimensions encode either angles of a hand pose or the state of the gripper, with 0 representing closed grippers and 1 representing open grippers. Dimensions that are unused for an embodiment are zero padded. Note that some robot datasets rely on a separate 14 dimensional action space, split into 7 dimension for left/primary and 7 dimensions for right/secondary arms. Here, we train directly on 6 joint states and one gripper state as we do not have models of the forward dynamics of the embodiment. Across all action datasets, action rates range from 5 Hz to 30 Hz. For the EE50 action space, the end effector pose is represented relative to an anchor frame, which is the last conditioning frame. We choose the action state closest to this anchor frame as the anchor state, and represent all other end effector poses relative to this state. Hand articulation and gripper values remain absolute. Finally, we normalize each EE50 dimension using embodiment specific z-scores as in Punamiya et al. 2025<sup 2 R 19bdbsnpfcpflb -note-1 </sup indexed by the signed position offset from anchor state. With the EE50 action head, we can now also evaluate RoboLab performance of midtraining checkpoints directly without additional finetuning. The results of these evaluations are shown with hatched lines in Figure 4. Initially, we observe that the EE50 performance quickly increases from 0% over 7.2% to 14.7% with modest gains for the results obtained with Joint finetuning - which improves only 2.3% over its performance in the last pretraining checkpoint. Towards later steps, we see continued improvements for both EE50, reaching 17.3%, and Joint finetuning performance, reaching 18.6%. We have not yet seen performance saturating in midtraining. Noise-to-Signal Transition for Videos and Actions Diffusion and flow models are learning to reverse a time-indexed forward process z<sub t</sub = 1−t x₀ + tε that mixes data x₀ with noise ε ∼ N 0,1 . Depending on the scale and structure of the data, the region of t where this forward process transitions between noise and signal can be different. During training, t is sampled from a timestep distribution, and its choice can affect the performance of models significantly<sup 1 R abdbsnpfcpflb -note-0 </sup . When jointly predicting two modalities such as videos and actions, we also have to consider the interplay between the two modalities. To get a sense of the regions of signal-to-noise transition for the two modalities, we compute the x₀ reconstruction loss separately for both. We do this for multiple checkpoints trained with various timestep distributions and then take the pointwise minimum over the checkpoints to obtain the minimum-envelope x₀ loss as an approximation for the signal-to-noise profile of videos and actions. We plot these envelopes, normalized to a maximum loss of 1, in Figure 5. We observe an S-shaped profile with a transition around σ t = 3.3 or t ≃ 0.964. Assuming that model capacity is spent best in regions where the signal transition is high, a reasonable approach is to use a timestep distribution whose probability density function pdf is proportional to the derivative of the S-shaped profile. Depending on the function we use to approximate the S-shape, we end up with different timestep distributions. When using a sigmoid function for the approximation as shown in Figure 5, we end up with a logistic distribution in logit space, i.e. the logit-logistic distributions shown in Figure 6. Similarly, when using a Gaussian CDF, we end up with a Gaussian distribution in logit space or a logit-normal distribution. We use these estimates as guides for choosing timestep distributions in a grid search. Besides logit-normal and logit-logistic distributions, we also include the “mode-sampler” or “Waver”<sup 2 R 1abdbsnpfcpflb -note-0 </sup timestep distribution as it is the choice in NVIDIA et al. 2026<sup 3 R 1abdbsnpfcpflb -note-1 </sup . We formulate the location parameter for all three candidate distributions through a zero-centered base distribution followed by an application of the shifting function, t ↦ αt / 1 + α−1 t , with parameter α. For logit-normal and logit-logistic distributions, this results in a shift of their mode to logit t = log α. We choose α ∈ {24, 33, 42} corresponding to the modes of the video and action estimates in Figure 6 together with one value in-between. We use unit variance for the logit-normal distribution and consider logit-logistic distributions with scale 0.5 and 0.75. For the Waver distribution, we follow NVIDIA et al. 2026 and use a shift of α = 5 and sweep over one higher 10 and one lower 2.5 value. Besides the timestep distribution itself, we consider one additional parameter: the scale of the action representation not to be confused with the scale of the timestep distribution . When multiplying actions by s, but not videos, we shift the relative signal-noise transition in Figure 5 between the two. When s 1, more noise is required to destroy the signal and the action curve gets shifted to the right. This allows us to control how different modalities are modeled in relationship to one another. Consider two modalities m₁, m₂. Assuming their signal-noise profiles are roughly matched, a model for m₁, s·m₂ with s ≫ 1 would effectively approximate a factorization like p m₁, s·m₂ = p m₁ | s·m₂ p s·m₂ under a joint flow matching model, because during the transition of s·m₂ from noise to signal, m₁ would still be mostly noise but whenever we get to the transition point for m₁, s·m₂ is already mostly signal. Similarly, s ≪ 1 would achieve an approximation of p m₁, s·m₂ = p s·m₂ | m₁ p m₁ . To explore the effect of this, we consider scaling factors s ∈ {1, 2} for the action modality. To control for effects on the loss weighting, we reweight the loss on the action modality by 1/s². We train the different variants and report results across multiple seeds on the simple task suite of RoboLab in Figure 7. Overall, we fail to identify a clear trend in these results, but we consistently observe high performance for logit-logistic distributions with an action scaling of s = 2 and high shifts of α = 42. Among the two scales of the logit-logistic distribution, the wider one with scale 0.75 performs best and we continue with this setting. DROID Recipe When finetuning on DROID, we add randomly initialized action heads to our midtrain checkpoint. Thus, the finetuning will begin with a very noisy gradient signal which could affect the model negatively. To ensure that the midtraining weights are not degraded during this initial adaptation phase, we keep the midtraining weights frozen for the first 1k steps, and then linearly increase their learning rate from 0 to 2e-4 over 2k steps. The learning rate for the action heads themselves warms up linearly from 0 to 1e-3 over the first 1k steps. We compare this strategy to a run where we finetune DROID with end effector poses adapted to our EE50 action space see Pretraining and Midtraining . For this variant, the action heads are already initialized from midtraining and we use a simple learning rate warmup from 0 to 2e-4 over the first 1k steps for all parameters. We train both configurations with two batch sizes, 512 and 2048, for 20k steps. For the small batch size configuration, EE50 performance remains above the Joint finetuning performance. Performance scales well with batch size and only incurring a small cost when comparing under matched FLOPs: the performance of the EE50 and Joint runs with 4x larger batch size at step 5k is only degraded by 2.25 pp and 4.25 pp, respectively, compared to the smaller batch size run after 20k steps. However, after 10k steps with the larger batch size, Joint finetuning outperforms EE50 finetuning with a final performance of 40.13% against 37.88%. We continue with the Joint finetuning run but note that EE50 finetuning can be an attractive option for finetuning with very little data or training steps - especially considering that the timestep distribution was chosen based on its performance for Joint finetuning. To understand if performance is saturated, we used the following protocol for choosing the best checkpoint: we continue training the Joint run for a total of 30k steps and use a linear learning rate cooldown in the last 5k steps. Then we evaluate every 1k steps during the cooldown and select the best performing checkpoint from this. Although the training loss shows continued improvements, especially during the cooldown phase, we observe no additional improvements beyond 20k steps of finetuning. The best performing one from the cooldown phase reaches 40% at 26k steps. With higher LRs, in the vicinity of 2e-4 base LR and above, we observed sporadic loss spikes. As illustrated in Figure 10, training affected by a loss spike has lower RoboLab success rates compared to when resumed from a prior clean checkpoint. Despite the success rates eventually recovering here, we observed instances where the runs take much longer to recover, leading to subpar performance within the training horizon. Hence, we always rolled back after observing loss spikes to maintain comparable checkpoints and avoid adversely affecting final performance. Split Guidance for Video and Action Given that inference of an action chunk predicts both future video as well as future actions, we can do classifier-free guidance<sup 1 R cbdbsnpfcpflb -note-0 </sup independently on each modality. We analyze the CFG behavior by running a grid-search over {Video CFG, Action CFG} hyperparameters and evaluating success rates on the full 120 task suite. We observe a gradient trend that prefers higher Video CFG and lower Action CFG, with Video CFG = 4.0 and Action CFG = 1.0 performing the best with a score of 42.00%. We choose this setting as our preferred inference for production setting. EMA Analysis We observed EMA to be critical both for performance and for reliably comparing ablation sweeps. Hence, we opted to keep track of both traditional EMA and Power EMA<sup 1 R dbdbsnpfcpflb -note-0 </sup σ<sub rel</sub = 0.05 and 0.10 . For traditional EMA, the update step is where θ t are the raw parameters at training step t. In our experiments, we chose β = 0.9990. Since we log checkpoints every K = 1000 steps, we also evaluated “offline” EMA variants for other β values. The average is taken over the finite window of logged snapshots and normalized by the weight actually accumulated within it, where n indexes the N logged checkpoints. Raising β to the power K makes the per-checkpoint recursion match the per-step profile of Equation 1, up to the loss of intra-interval detail, and the denominator removes the start-up bias of the finite window. For Power EMA, the update step is which is computed incrementally as where t is the one-indexed update count. Following Karras et al. 2024 , we parameterize the profile by its relative width σ<sub rel</sub = γ+1 <sup 1/2</sup γ+2 <sup −1</sup γ+3 <sup −1/2</sup . We track two variants during training: one with σ<sub rel</sub = 0.05 and the other with σ<sub rel</sub = 0.10 equivalently, γ = 16.9722 and γ = 6.9372 . With the two profiles, we can perform a more accurate post-hoc reconstruction for new σ<sub rel</sub values, where p<sub γ,t</sub τ denotes the weighting profile that Equation 3 assigns to θ τ when averaged up to time t. The basis consists of both tracked profiles at every logged checkpoint, i.e. the 2N stored snapshots {θ̂<sub γᵢ</sub t<sub n</sub }. The fit is unconstrained, so coefficients may be negative, and x̃ is normalized to sum to one, making the reconstruction an affine combination of the snapshots. The results for the various EMA settings are summarized in Figure 12. The traditional online EMA and both Power EMA models healthily outperform the raw model, showcasing the importance of EMA. We also observe that Power EMA edges out traditional EMA. For the post-hoc Power EMA and offline EMA candidates, we observe a bell-shaped trend, with success rates peaking at σ<sub rel</sub = 0.150 and β = 0.9997, respectively. However, in both cases, online averaging, where updates happen every step, is stronger than the post-hoc approximation. Considering the full picture, we observe that online Power EMA with σ<sub rel</sub = 0.10 consistently performs best, and we choose it as our recommended EMA setting for the DROID recipe. Inference Efficiency WAMs have shown promising results in data efficiency and performance. However, predicting not only actions but also videos makes them comparatively slow due to their larger sequence lengths compared to VLAs. This limits their application in latency critical situations or when working under hardware constraints such as local deployments. Previous approaches mostly relied on system level optimizations or a decoupling of actions and videos through separate decoders or attention masking. Here, we consider an orthogonal direction and explore how close we can get to the efficiency of VLAs with distillation while keeping joint prediction of actions and videos in place. Our first approach is guidance distillation. As we have seen in Split Guidance for Video and Action, guidance significantly improves performance. However, it introduces the need for a second, unguided, forward pass for each sampling step. If a single forward pass already utilizes parallel capacity of the hardware sufficiently, this increases latency roughly by a factor of 2x. System level optimizations hide this additional latency by distributing the two forward passes across two independent GPUs but this increases hardware costs by a factor of 2x. Instead of parallelizing the two forward passes, we distill the guided prediction of the model into a guidance distilled student model, F3A DROID GD: We initialize student and teacher from our F3A DROID checkpoint. The teacher is kept frozen and provides its guided prediction on DROID examples, which serves as the target for the student training. After training, we run the student inference without guidance which gives a speedup between 1.8x - 2x. Performance is not only retained, but actually improves performance between 0.6 - 1.08 pp. The next target for further improvements is step distillation. Here, we initialize student and teacher from the guidance distilled checkpoint and train the student for single step prediction under guidance of the teacher. By reducing the number of sampling steps by 4x, we obtain expected speedups in the range of 3.15 - 4x. While this represents a drastic speed improvement, it does come with a reduction of -3.51 - -4.32 pp in success rate, although this still remains 1.12 - 1.88 pp above Cosmos 3 performance, while being 9.19 - 13.22x faster. Compared to pi0.5, F3A SD improves performance by 9.92 - 10.68 pp while the comparison of real-time factor, RTF = Processing time / Chunk duration, depends on the hardware. In FP8, F3A SD provides a 1.34x through 2.28x speedup over Pi0.5 in BF16 on workstation and datacenter GPUs, while on consumer GPUs such as RTX 5090 it is 1.15 - 2.42x worse in FP8 and BF16, respectively. For inference, we optimize by pruning unnecessary computation, running in FP8, and leveraging torch.compile with reduced-overhead for minimal latency. We observe little effect on task success rates by using rowwise dynamic FP8, ranging between small +0.05 pp improvements for the guidance distilled checkpoint to -0.76 pp reduction for the step distilled checkpoint, while providing over 1.4x speedup on local GPUs RTX Pro, 5090 . Further, during inference, the prompt tokens do not change in the mode blocks of the architecture, so the latter can be cached once whenever the prompt changes. Similarly, the shapes are static for a given canvas even if the observations change. This enables efficient compilation with overhead reduction. The last piece is efficient computation, moving step invariant and shared computation out of the blocks and into joint preparation steps. All results can also be found in Figure 1 and the tables below. | Policy | Actions per prediction | Control rate Hz | Full-chunk duration s | |---|---|---|---| | Pi0.5 | 15 | 15 | 1.00 | | F3 + C3 | 32 | 15 | 2.13 | | Workload | BF16 ms | FP8 ms | |---|---|---| | F3 · 4 steps · guidance on | 246.42 | 182.00 | | F3 · 4 steps · guidance off | 136.25 | 101.71 | | F3 · 1 step · guidance off | 41.06 | 32.29 | | Cosmos 3 Nano · 4 steps · guidance on | 387.71 | 320.40 | | π0.5 · 10 steps · guidance off | 31.99 | Not evaluated | | Setting | Seeds | Mean ± SEM % | |---|---|---| | Base BF16 | 6 | 41.60± 0.46Seeds42.6740.2540.4242.9241.4241.92 | | Base FP8 | 8 | 41.16± 0.30Seeds41.4240.3341.5840.8342.4241.4239.6741.58 | | Guidance Distilled BF16 | 6 | 42.19± 0.39Seeds40.6741.5042.7542.3342.6743.25 | | Guidance Distilled FP8 | 6 | 42.24± 0.36Seeds43.0841.6743.0042.1742.6740.83 | | Step Distilled BF16 | 6 | 38.68± 0.29Seeds37.9238.2538.0039.2539.5039.17 | | Step Distilled FP8 | 6 | 37.92± 0.31Seeds38.0036.9239.0838.2537.9237.33 | Real Robot Evals We carried out independent third-party evaluations of our action policy. Positronic Robotics https://positronic.ro/ ran our action policy on a Franka arm in its lab, on ten DROID tasks with three attempts per task, alongside Cosmos3-Nano, DreamZero and pi0.5. Every model got the same tasks, the same setup, the same 240-second window per attempt and the same scoring. The operator did not know which model drove each attempt, and every attempt was recorded on video. Our policy successfully completed 28 of its 30 attempts. An overview of the metrics is provided in Table 6. A few successful rollouts are shown in Figure 13, Figure 14, Figure 15, and Figure 16. We also applied the same finetuning recipe on the SO-101 embodiment. We collected a small dataset using teleoperation and finetuned the model on top of it. We showcase the model’s rollouts for in-distribution tasks in Figure 17 and its ability to generalize to object, container, and camera changes that are out-of-distribution in Figure 18, Figure 19, and Figure 20, respectively. | Task | F3A | Cosmos 3 Nano | DreamZero | π0.5 | |---|---|---|---|---| | Sponge in bowl | | | | | | Put cube in bowl | | | | | | Yellow block in blue cup | | | | | | Put fork on plate | | | | | | Banana on left plate | | | | | | Marker in bowl | | | | | | Cup in bowl | | | | | | Yellow block in cup | | | | | | Move cup left | | | | | | Close the drawer | | | | | | Overall | 93.3% 28/30 | 90.0% 27/30 | 66.7% 20/30 | 43.3% 13/30 | Hybrid policies F3A provides a fast system for embodied action prediction, producing robot trajectories in as little as a single forward pass on consumer hardware. It can solve a broad range of manipulation tasks and improves performance over previous systems. But some tasks require capabilities beyond intuitive control, such as decomposing complex tasks with a long horizon or making a multi-step plan. Reasoning agents such as GPT 6 Astra provide such capability and recently showed impressive control behavior<sup 1 R 1fdbsnpfcpflb -note-0 </sup . However, it remains impractical to run them directly as policies, with reasoning calls on average taking around 16 seconds. Nevertheless, Su et al. 2026 <sup 2 R 1fdbsnpfcpflb -note-1 </sup demonstrated first promising results of combining GPT 6 Astra with Pi0.5. Here, we ask whether such a combination improves the efficiency of embodied reasoning, and whether improvements in action policies - as we obtained them with F3A - translate to improvements in this efficiency. To answer this question, we replicate the setup of Su et al. 2026 : For each episode, GPT 6 Astra runs as an agent that keeps track of the episode’s history. In each turn, the action policy - F3A or pi0.5 in our experiments - makes a prediction based on the current observations and state. The joint prediction coming out of this gets converted to end-effector pose through forward kinematics. This result, along with observations and proprioceptive state, is then made available to Astra. Subsequently, it can decide between one of four options: execute a subset of the policy’s predicted actions, modify between 1-5 of the policy’s actions, propose its own actions or stop. The resulting actions are executed through inverse kinematics in RoboLab and the cycle repeats. We also include a pure variant, where Astra receives no action prediction from another policy and instead always has to make its own prediction. Note that we remained as close as possible to Su et al. 2026 and were able to reproduce their reported results. We collected timings and token counts for all Astra calls to estimate efficiency and extend their results by including F3A and running all variants across three different reasoning efforts of Astra low, medium, xhigh . From this data, we can give a more concrete sense of the drastic efficiency difference between pure action policies such as F3A and pure embodied reasoning agents such as GPT 6 Astra: From Table 7, we find that the most cost effective Astra variant uses xhigh efforts to achieve a cost of $13.47 per success. This variant requires an average of 16m 23s per success, whereas low reasoning efforts improve this to 14m 21s. Now, to get comparable estimates for F3A, our FP8 SD checkpoint executed a total of 147802 steps computed in 4741 policy calls for 96 successes across two seeds. Thus, on a H200 we end up with a total time of 104.8s per successful call, although most of this time is spent idle by the policy as the actions get executed. With a latency of 43.81ms on the H200, it could in theory operate 48.62 rollouts at the same time resulting in an effective time of 2.16s per success. Summarized, in a single rollout scenario with a mostly idle policy, it takes around 1.75 minutes and $0.087 per success when running on an H200 with an hourly price of $3.00/GPU-h, which makes it 8.57x more time and 155x more cost effective than the most cost effective pure Astra variant. When serving multiple environments in parallel, time per success reduces to 2.16s with a cost of $0.0018 per success - an improvement of almost 400x in time efficiency and 7843x in cost. However, it must be noted that pure Astra variants as well as hybrid variants significantly improve success rates, especially for more complex tasks, such as “Pumpkins in clutter” which none of the pure action policies can solve on their own currently. For the hybrid policies, we see that Pi0.5 can only bring modest improvements in efficiency, reducing costs to $12.28 with medium reasoning efforts and time to 13m 33s with low reasoning efforts. F3A on the other hand improves efficiency across all reasoning levels and reduces costs to $8.77 and time to 8m 08s per success in the low setting, which correspond to an improvement of 2x and 1.54x while retaining a high success rate of 90% and solving tasks of all complexities. One of the tasks that pure action policies struggle with is “Stack the blocks in the order from bottom to top: red, blue, green, yellow”. In the video below, we show a total of three rollouts: in the top two rollouts from F3A on its own without Astra involvement, and in the bottom a rollout from the hybrid policy of F3 + Astra low , where the overlay indicates whether F3A or Astra is in control at any given moment. We see that F3A does have the visuomotoric skills necessary to stack the blocks - however it seems to ignore the requested ordering. In the hybrid policy at the bottom, we see that Astra intervenes for a short duration, where F3A was about to pick up the yellow block to stack it on top of the already stacked red and blue blocks. A short correction of moving the end-effector closer to the green block was sufficient to let F3A continue again on its own. | Configuration | Success rate | Cost per success $ | Time per success | Tokens per success | |---|---|---|---|---| | F3 + Astra xhigh | 94% 47/50 | 11.00 | 11m 53s | 7.83M | | Pi0.5 + Astra xhigh | 92% 46/50 | 14.41 | 17m 09s | 10.12M | | F3 + Astra medium | 84% 42/50 | 10.41 | 10m 05s | 7.69M | | Pi0.5 + Astra medium | 86% 43/50 | 12.28 | 14m 26s | 8.86M | | F3 + Astra low | 90% 45/50 | 8.77 | 8m 08s | 6.60M | | Pi0.5 + Astra low | 86% 43/50 | 15.57 | 13m 33s | 12.38M | | Pure Astra xhigh | 100% 50/50 | 13.47 | 16m 23s | 10.00M | | Pure Astra medium | 94% 47/50 | 15.50 | 14m 48s | 12.11M | | Pure Astra low | 84% 42/50 | 17.77 | 14m 21s | 14.13M | Full results | Configuration | Cost / success $ | Time / success | Successes | Time | Tokens | Cost $ | Tokens / success | |---|---|---|---|---|---|---|---| | F3 + Astra low | 8.77 | 8m 08s | 45/50 | 6:05:45 | 296.95M | 394.69 | 6.60M | | F3 + Astra medium | 10.41 | 10m 05s | 42/50 | 7:03:30 | 323.17M | 437.28 | 7.69M | | F3 + Astra xhigh | 11.00 | 11m 53s | 47/50 | 9:18:16 | 368.00M | 517.08 | 7.83M | | Pi0.5 + Astra medium | 12.28 | 14m 26s | 43/50 | 10:20:58 | 380.82M | 528.00 | 8.86M | | Pure Astra xhigh | 13.47 | 16m 23s | 50/50 | 13:39:13 | 500.15M | 673.66 | 10.00M | | Pi0.5 + Astra xhigh | 14.41 | 17m 09s | 46/50 | 13:08:42 | 465.70M | 662.87 | 10.12M | | Pure Astra medium | 15.50 | 14m 48s | 47/50 | 11:35:57 | 569.20M | 728.55 | 12.11M | | Pi0.5 + Astra low | 15.57 | 13m 33s | 43/50 | 9:42:45 | 532.30M | 669.46 | 12.38M | | Pure Astra low | 17.77 | 14m 21s | 42/50 | 10:02:34 | 593.64M | 746.32 | 14.13M | | | F3 + Astra | | | Pi0.5 + Astra | | | Pure Astra | | | Reported | | | | | |---|---|---|---|---|---|---|---|---|---|---|---|---|---|---| | Task | low | medium | xhigh | low | medium | xhigh | low | medium | xhigh | Pi0.5 + Astra | F3 | Pi0.5 | Cosmos 3 | DreamZero | | Blocks into bin | | | | | | | | | | | 3/20 | | | | | Pumpkins in clutter | | | | | | | | | | | 0/20 | | | | | Butter on raisin box | | | | | | | | | | | 19/20 | | | | | Stack blocks in order | | | | | | | | | | | 0/20 | | | | | Reorient red mug | | | | | | | | | | | 16/20 | | | | | Larger raisin box into bin | | | | | | | | | | | 2/20 | | | | | Sauce bottle into crate | | | | | | | | | | | 20/20 | | | | | Canned food into bin | | | | | | | | | | | 15/20 | | | | | Yogurt into bowl | | | | | | | | | | | 1/20 | | | | | Rubik's cube into bowl | | | | | | | | | | | 20/20 | | | | | Overall | 90% 45/50 | 84% 42/50 | 94% 47/50 | 86% 43/50 | 86% 43/50 | 92% 46/50 | 84% 42/50 | 94% 47/50 | 100% 50/50 | 92% 46/50 | 48% 96/200 | 36% 18/50 | 36% 18/50 | 34% 17/50 | Outlook Action prediction offers a direct way for agents to interact with the world. To translate this capability into productivity improvements in the physical world, action policies must become easier and more efficient to deploy by learning new tasks from a few demonstrations in a manner that they can subsequently solve them reliably in real-time. However, when only considering opportunities for industrial automation, the challenges we can evaluate at scale on physical robots - manipulation tasks in controlled settings - is just a small slice of visual intelligence. Capabilities such as navigating unfamiliar territory toward a goal, picking out what barely separates from its background, anticipating how other agents will act and adjusting to them, developing strategies and planning ahead under partial information can’t be evaluated safely or at scale in the real world. Games on the other hand were designed over millions of man hours to challenge these and many more capabilities. In contrast to a physical robot, they run in parallel, faster than real time, and without risk. As such, they provide an ideal testbed for research on action prediction and its deeper integration with reasoning capabilities - and an agent that plays games is just one short step from operating other software. We hope that our open weight release allows more people to explore this direction. Shout out to mimic https://www.mimicrobotics.com/ for many helpful discussions and sharing of knowledge, to Positronic Robotics https://positronic.ro/ for a speedy evaluation of our policy on their platform, to NVIDIA for help with integrating our model on NVIDIA Jetson, HuggingFace and LeRobot as well as testing parameter efficient fine-tuning on SO-101, and to the NVIDIA Isaac Lab and RoboLab teams, which provided the basis for most of our evaluations. The model was trained on NVIDIA GB200 systems, with custom kernels written in NVIDIA’s CuTe DSL. References 1. Cen, Jun, Chaohui Yu, Hangjie Yuan, et al. 2025. WorldVLA: Towards Autoregressive Action World Model. arxiv.org/abs/2506.21539 https://arxiv.org/abs/2506.21539 2. Cheang, Chi-Lam, Guangzeng Chen, Ya Jing, et al. 2024. GR-2: A Generative Video-Language-Action Model with Web-Scale Knowledge for Robot Manipulation. arxiv.org/abs/2410.06158 https://arxiv.org/abs/2410.06158 3. Chefer, Hila, Patrick Esser, Dominik Lorenz, et al. 2026. Self-Supervised Flow Matching for Scalable Multi-Modal Synthesis. arxiv.org/abs/2603.06507 https://arxiv.org/abs/2603.06507 4. Chen, Xiao. 2026. Awesome Astra Embodied AI. GitHub repository. github.com/zjwzcx/Awesome-Astra-Embodied-AI https://github.com/zjwzcx/Awesome-Astra-Embodied-AI 5. Du, Yilun, Mengjiao Yang, Bo Dai, et al. 2023. Learning Universal Policies via Text-Guided Video Generation. arxiv.org/abs/2302.00111 https://arxiv.org/abs/2302.00111 6. Ebert, Frederik, Chelsea Finn, Sudeep Dasari, Annie Xie, Alex Lee, and Sergey Levine. 2018. Visual Foresight: Model-Based Deep Reinforcement Learning for Vision-Based Robotic Control. arxiv.org/abs/1812.00568 https://arxiv.org/abs/1812.00568 7. Esser, Patrick, Sumith Kulal, Andreas Blattmann, et al. 2024. Scaling Rectified Flow Transformers for High-Resolution Image Synthesis. arxiv.org/abs/2403.03206 https://arxiv.org/abs/2403.03206 8. Finn, Chelsea, Ian Goodfellow, and Sergey Levine. 2016. Unsupervised Learning for Physical Interaction Through Video Prediction. arxiv.org/abs/1605.07157 https://arxiv.org/abs/1605.07157 9. Finn, Chelsea, and Sergey Levine. 2016. Deep Visual Foresight for Planning Robot Motion. arxiv.org/abs/1610.00696 https://arxiv.org/abs/1610.00696 10. Fragkiadaki, Katerina, Pulkit Agrawal, Sergey Levine, and Jitendra Malik. 2015. Learning Visual Predictive Models of Physics for Playing Billiards. arxiv.org/abs/1511.07404 https://arxiv.org/abs/1511.07404 11. Hang, Tiankai, Shuyang Gu, Chen Li, et al. 2024. Efficient Diffusion Training via Min-SNR Weighting Strategy. arxiv.org/abs/2303.09556 https://arxiv.org/abs/2303.09556 12. Ho, Jonathan, and Tim Salimans. 2022. Classifier-Free Diffusion Guidance. arxiv.org/abs/2207.12598 https://arxiv.org/abs/2207.12598 13. Hoogeboom, Emiel, Jonathan Heek, and Tim Salimans. 2023. Simple Diffusion: End-to-End Diffusion for High Resolution Images. arxiv.org/abs/2301.11093 https://arxiv.org/abs/2301.11093 14. Intelligence, Physical, Kevin Black, Noah Brown, et al. 2025. π0.5: A Vision-Language-Action Model with Open-World Generalization. arxiv.org/abs/2504.16054 https://arxiv.org/abs/2504.16054 15. Karras, Tero, Miika Aittala, Timo Aila, and Samuli Laine. 2022. Elucidating the Design Space of Diffusion-Based Generative Models. arxiv.org/abs/2206.00364 https://arxiv.org/abs/2206.00364 16. Karras, Tero, Miika Aittala, Jaakko Lehtinen, Janne Hellsten, Timo Aila, and Samuli Laine. 2024. Analyzing and Improving the Training Dynamics of Diffusion Models. arxiv.org/abs/2312.02696 https://arxiv.org/abs/2312.02696 17. Kim, Moo Jin, Yihuai Gao, Tsung-Yi Lin, et al. 2026. Cosmos Policy: Fine-Tuning Video Models for Visuomotor Control and Planning. arxiv.org/abs/2601.16163 https://arxiv.org/abs/2601.16163 18. Kingma, Diederik P., Tim Salimans, Ben Poole, and Jonathan Ho. 2023. Variational Diffusion Models. arxiv.org/abs/2107.00630 https://arxiv.org/abs/2107.00630 19. Li, Lin, Qihang Zhang, Yiming Luo, et al. 2026. Causal World Modeling for Robot Control. arxiv.org/abs/2601.21998 https://arxiv.org/abs/2601.21998 20. Li, Shuang, Yihuai Gao, Dorsa Sadigh, and Shuran Song. 2025. Unified Video Action Model. arxiv.org/abs/2503.00200 https://arxiv.org/abs/2503.00200 21. Nichol, Alex, and Prafulla Dhariwal. 2021. Improved Denoising Diffusion Probabilistic Models. arxiv.org/abs/2102.09672 https://arxiv.org/abs/2102.09672 22. NVIDIA. 2026. Cosmos 3: Omnimodal World Models for Physical AI. arxiv.org/abs/2606.02800 https://arxiv.org/abs/2606.02800 23. Oh, Junhyuk, Xiaoxiao Guo, Honglak Lee, Richard Lewis, and Satinder Singh. 2015. Action-Conditional Video Prediction Using Deep Networks in Atari Games. arxiv.org/abs/1507.08750 https://arxiv.org/abs/1507.08750 24. Pai, Jonas, Liam Achenbach, Victoriano Montesinos, Benedek Forrai, Oier Mees, and Elvis Nava. 2025. Mimic-Video: Video-Action Models for Generalizable Robot Control Beyond VLAs. arxiv.org/abs/2512.15692 https://arxiv.org/abs/2512.15692 25. Punamiya, Ryan, Dhruv Patel, Patcharapong Aphiwetsa, et al. 2025. EgoBridge: Domain Adaptation for Generalizable Imitation from Egocentric Human Data. arxiv.org/abs/2509.19626 https://arxiv.org/abs/2509.19626 26. Raya, Gabriel, Bac Nguyen, Georgios Batzolis, et al. 2026. Noise Scheduling as Information-Guided Allocation in Diffusion Training. arxiv.org/abs/2602.18647 https://arxiv.org/abs/2602.18647 27. Schmidhuber, Jürgen. 1990. Making the World Differentiable: On Using Self-Supervised Fully Recurrent Neural Networks for Dynamic Reinforcement Learning and Planning in Non-Stationary Environments. Technical Report FKI-126-90, Institut für Informatik, Technische Universität München. 28. Su, Jiayi, Yixin Zheng, Mi Yan, Li Yi, Zhizheng Zhang, and He Wang. 2026. GPT 6 Astra as an Embodied Policy. Technical report and code. github.com/anonymous-report-421/GPT-as-Policy https://github.com/anonymous-report-421/GPT-as-Policy 29. Wu, Hongtao, Ya Jing, Chilam Cheang, et al. 2023. Unleashing Large-Scale Video Generative Pre-Training for Visual Robot Manipulation. arxiv.org/abs/2312.13139 https://arxiv.org/abs/2312.13139 30. Yang, Jenai Xuning, Rishit Dagli, Alex Zook, et al. 2026. RoboLab: A High-Fidelity Simulation Benchmark for Analysis of Task Generalist Policies. arxiv.org/abs/2604.09860 https://arxiv.org/abs/2604.09860 31. Ye, Angen, Boyuan Wang, Chaojun Ni, et al. 2026. GigaWorld-Policy: An Efficient Action-Centered World-Action Model. arxiv.org/abs/2603.17240 https://arxiv.org/abs/2603.17240 32. Ye, Seonghyeon, Yunhao Ge, Kaiyuan Zheng, et al. 2026. World Action Models Are Zero-Shot Policies. arxiv.org/abs/2602.15922 https://arxiv.org/abs/2602.15922 33. Yuan, Tianyuan, Zibin Dong, Yicheng Liu, and Hang Zhao. 2026. Fast-WAM: Do World Action Models Need Test-Time Future Imagination? arxiv.org/abs/2603.16666 https://arxiv.org/abs/2603.16666 34. Zhang, Yifu, Hao Yang, Yuqi Zhang, et al. 2025. Waver: Wave Your Way to Lifelike Video Generation. arxiv.org/abs/2508.15761 https://arxiv.org/abs/2508.15761 35. Zhou, Yi, Connelly Barnes, Jingwan Lu, Jimei Yang, and Hao Li. 2019. On the Continuity of Rotation Representations in Neural Networks. 36. Zhu, Chuning, Raymond Yu, Siyuan Feng, Benjamin Burchfiel, Paarth Shah, and Abhishek Gupta. 2025. Unified World Models: Coupling Video and Action Diffusion for Pretraining on Large Robotic Datasets. arxiv.org/abs/2504.02792 https://arxiv.org/abs/2504.02792