# The HydroGym reinforcement learning platform for fluid dynamics > Source: > Published: 2026-09-08 08:31:05+00:00 ## Abstract Effective control of fluid flows is critical across transportation, energy and medicine, where it can increase lift, reduce drag, enhance mixing and attenuate noise[1](#ref-CR1),[2](#ref-CR2),[3](/articles/s41586-026-10917-6#ref-CR3). Yet fluids are notoriously difficult to control because they involve high-dimensional, nonlinear and multiscale dynamics that resist conventional approaches[4](#ref-CR4),[5](#ref-CR5),[6](/articles/s41586-026-10917-6#ref-CR6). Reinforcement learning has driven remarkable progress in fields such as protein folding and complex games, which have shared benchmarks and standardized environments[7](#ref-CR7),[8](#ref-CR8),[9](#ref-CR9),[10](/articles/s41586-026-10917-6#ref-CR10). Fluid dynamics has lacked such infrastructure, so each controller is typically tuned to a single geometry and operating condition, making progress difficult to accumulate, transfer and compare[11](#ref-CR11),[12](#ref-CR12),[13](/articles/s41586-026-10917-6#ref-CR13). Here we introduce HydroGym, a solver-independent reinforcement learning platform providing more than 60 validated, openly available flow control environments spanning from canonical laminar flows to complex turbulent flows, with systematic progression in the Reynolds number up to Re = 4 × 105, and Mach number variations in two and three dimensions. Across these environments, agents repeatedly discover robust control principles, including boundary layer manipulation, disruption of acoustic feedback and reorganization of turbulent wakes. Critically, we demonstrate a proof of concept for zero-shot transfer, in which agents that are trained exclusively in inexpensive surrogate environments are deployed to challenging real-world scenarios such as a three-dimensional wing section. We achieve a 38% reduction in local skin friction while reducing exploration costs by four orders of magnitude compared with direct on-wing optimization. As this transfer exploits shared near-wall physics, the breadth of generalization remains open, suggesting a new pathway for research toward policy generalization across computationally prohibitive simulation environments. By offering a common, extensible foundation for reproducible research, HydroGym moves flow control from isolated case studies toward a cohesive community effort. ### Similar content being viewed by others ## Main Fluid flow control represents a critical challenge in several trillion-dollar industries including transportation (for example, aerospace, automotive, shipping), energy production (wind farms, gas turbines, pipelines), manufacturing (spray processes, additive manufacturing, thin films and coatings) and medicine (haemodynamic, respiratory and lymphatic systems). Advanced control systems could dramatically improve energy efficiency globally, for example reducing fuel consumption in the aviation industry by up to 15% through active drag reduction[1](#ref-CR1),[2](#ref-CR2),[3](/articles/s41586-026-10917-6#ref-CR3),[14](/articles/s41586-026-10917-6#ref-CR14),[15](/articles/s41586-026-10917-6#ref-CR15), or increasing wind farm output by 4–5% through coordinated control[16](#ref-CR16),[17](#ref-CR17),[18](/articles/s41586-026-10917-6#ref-CR18). Despite decades of progress in traditional control approaches, the nonlinear, multiscale nature of turbulence results in control problems that are high-dimensional, non-convex and computationally intractable using conventional methods[4](#ref-CR4),[5](#ref-CR5),[6](/articles/s41586-026-10917-6#ref-CR6),[19](/articles/s41586-026-10917-6#ref-CR19). Recent breakthroughs in Reinforcement Learning (RL) have demonstrated transformative capabilities across increasingly complex scientific applications, with AlphaFold revolutionizing protein structure prediction[7](/articles/s41586-026-10917-6#ref-CR7) and DeepMind achieving real-time control of fusion reactor plasmas[8](/articles/s41586-026-10917-6#ref-CR8). These successes share a common enabler: benchmark environments that democratize research access and accelerate algorithmic innovation. However, fluid dynamics has remained largely resistant to similar breakthroughs due to fundamental computational and accessibility barriers and a lack of comprehensive benchmark platforms for systematic algorithmic comparison and reproducible research. Training effective RL agents typically requires thousands or millions of interactions with the environment, and each evaluation in a fluid environment requires expensive computational fluid dynamics (CFD) simulations[11](/articles/s41586-026-10917-6#ref-CR11),[12](/articles/s41586-026-10917-6#ref-CR12),[20](/articles/s41586-026-10917-6#ref-CR20). This bottleneck is magnified by the inherent sample inefficiency of model-free RL[21](/articles/s41586-026-10917-6#ref-CR21),[22](/articles/s41586-026-10917-6#ref-CR22), creating severe computational limitations for large-scale flow control problems[13](/articles/s41586-026-10917-6#ref-CR13),[23](/articles/s41586-026-10917-6#ref-CR23),[24](/articles/s41586-026-10917-6#ref-CR24). Further, flow control research is currently caught in a specificity trap, where controllers trained at specific flow conditions may fail to generalize to different regimes. In much of today’s literature, each controller is carefully tuned to a single geometry, a specific operating point and a unique numerical set-up. Under these conditions, algorithmic progress and rigorous comparisons are nearly impossible, and generalization of control strategies across physical regimes remains a major challenge. We address these fundamental challenges in HydroGym—a reinforcement learning platform for flow control research. HydroGym establishes much-needed community infrastructure for common environments, interfaces and evaluation of algorithms, allowing competing methods to be tested fairly and extended to increasingly difficult regimes. Our solver-independent architecture provides sixty-one validated flow control environments spanning canonical laminar flows to advanced three-dimensional turbulent scenarios and Reynolds number progressions across flow regimes. The platform’s solver diversity helps bridge the disparate scales of fluid research: integrating established Python-based finite-element options for accessible prototyping; highly parallel GPU-accelerated lattice Boltzmann and finite-volume solvers for large-scale flows; and fully differentiable solvers to facilitate gradient-aware learning. Figure [1](/articles/s41586-026-10917-6#Fig1) shows the progression of HydroGym, from canonical environments of low-to-medium complexity, to highly challenging environments with complex three-dimensional geometries and fully turbulent flow physics at high Reynolds and Mach numbers. The HydroGym platform has several important flow classes, including incompressible and compressible high-Reynolds-number wall turbulence and airfoil environments for realistic deployment studies; shock-interaction regimes that isolate the physics of convective delays and non-local coupling; divergent nozzle configurations for strongly coupled compressible flows; and multi-agent settings for high-dimensional actuation (refer to Extended Data Fig. [1](/articles/s41586-026-10917-6#Fig5) for visualizations of all environments, and Extended Data Tables [1](/articles/s41586-026-10917-6#Tab1) and [2](/articles/s41586-026-10917-6#Tab2) for environment details including the Reynolds number, degrees of freedom, control objective and so on). Together, HydroGym establishes a community infrastructure to learn transferable physical control principles that generalize across geometries and flow regimes, towards increasingly relevant real-world systems. ## Systematic flow control demonstrations We discuss baseline performance metrics by considering four benchmark configurations using industry-standard RL algorithms: proximal policy optimization (PPO)[25](/articles/s41586-026-10917-6#ref-CR25), deep deterministic policy gradient (DDPG)[26](/articles/s41586-026-10917-6#ref-CR26), and twin delayed deep deterministic policy gradient (TD3)[27](/articles/s41586-026-10917-6#ref-CR27). In total, training of all validated baselines required over 150,000 GPU hours (Supplementary Section [4](/articles/s41586-026-10917-6#MOESM1)). Remarkably, the RL agents autonomously discover and exploit complex physical mechanisms such as boundary layer energization, feedback loop disruption and wake management through interaction alone. They successfully transform detrimental, separated flows into controlled, organized states while minimizing actuation, validating HydroGym’s utility for both fundamental physics research and practical control design. Results for presented 2D environments are provided in Extended Data Fig. [2](/articles/s41586-026-10917-6#Fig6), while the 3D demonstrations are detailed in Fig. [2](/articles/s41586-026-10917-6#Fig2). ### Fluidic pinball control Three cylinders in an equilateral formation create complex wake interactions, demonstrating coordinated multi-actuator control with fundamentally different dynamics across Reynolds numbers. At Re = 100 (2D), the uncontrolled system has natural flow asymmetry with irregular pressure oscillations. Reinforcement learning agents discover coordinated rotation strategies that achieve a drag reduction of approximately 90% by counter-rotating rear cylinders that inject momentum into the wake centreline, preventing shear layer interaction and suppressing vortex formation (Extended Data Fig. [2](/articles/s41586-026-10917-6#Fig6)). Increasing the Reynolds number to Re = 150, the flow undergoes a system bifurcation and becomes highly chaotic with irregular, unpredictable vortex shedding. The agents introduce an asymmetric forcing strategy on the front cylinder alongside counter-rotating rear cylinders. As a result, the rotation-induced Magnus forces create favourable pressure gradients, vectoring separated shear layers inward while energizing boundary layers. This transforms irregular, asymmetric shedding into a steady, symmetric wake with reduced velocity deficits and enhanced pressure recovery (Fig. [2](/articles/s41586-026-10917-6#Fig2)). Overall, this example demonstrates how targeted boundary layer manipulation can suppress complex dynamics in multi-body fluid systems. ### Open cavity flow control Cavity flows have complex dynamics dominated by shear layer instabilities and acoustic feedback, requiring control strategies that address multiple coupled timescales. At Re = 4,200, the 2D flow sits precisely at the threshold where small perturbations trigger transition from steady recirculating flow to unsteady vortex roll-up. The shear layer has weakly growing instabilities that manifest as low-amplitude pressure oscillations. Reinforcement learning agents discover that targeted momentum injection at the upstream cavity edge can exploit this marginal stability. The optimal policy uses controlled blowing to thicken the separating shear layer and reduce velocity gradients, suppressing Kelvin–Helmholtz instability growth before its amplification into coherent vortical structures. It exemplifies how small control inputs at near-critical conditions produce disproportionately large stabilization effects. In 3D at Re = 7,500, secondary flow structures emerge with well-developed acoustic feedback mechanisms. The shear layer develops coherent vortical structures that convect downstream and impinge on the cavity edge, generating pressure waves that propagate upstream to excite the separation point. This resonant feedback establishes dominant cavity tones and complex three-dimensional features including spanwise instabilities and vortex stretching. As shown in Fig. [2](/articles/s41586-026-10917-6#Fig2), RL agents discover sophisticated control policies using coordinated blowing and suction with temporal modulation. High-frequency, low-amplitude blowing at the upstream edge thickens the boundary layer and introduces controlled perturbations that interfere destructively with Kelvin–Helmholtz instabilities, while strategically placed suction near the downstream edge weakens the acoustic source by reducing vortex–edge interactions. The coordinated actuation achieves near-complete suppression of dominant cavity tones while controlling three-dimensional secondary flows, significantly reducing recirculation zone complexity and unsteady loading. ### Circular cylinder control at Re = 3,900 Moving to isolated bluff-body dynamics, the circular cylinder at Re = 3,900 (2D) features pronounced shear layer instabilities, irregular shedding frequencies and a chaotic wake structure. Operating under a zero-net-mass-flux (ZNMF) constraint, where opposing jets maintain equal and opposite mass flow rates, the controller implements boundary-layer manipulation that effectively streamlines the cylinder without adding or removing mass. The learned policy exploits asymmetric pressure distributions through coordinated suction and injection. Specifically, it pairs suction on one surface (removing low-momentum fluid to delay separation) with blowing on the opposite surface to energize the boundary layer through direct momentum injection. This creates a time-varying virtual geometry that actively manages wake formation. This localized boundary layer manipulation successfully transforms the chaotic wake into coherent, predictable vortical patterns that shed with improved regularity and symmetry (Extended Data Fig. [2](/articles/s41586-026-10917-6#Fig6)). The separation points shift downstream, narrowing the momentum deficit structure and converting random turbulent fluctuations into ordered momentum transport. ### Transverse gust mitigation at Re = 1,000 Finally, the gust mitigation environment presents a fundamentally different challenge requiring real-time adaptation. We consider a NACA0012 airfoil at a post-stall angle of attack (*α* = 20°) under transverse gusts with a gust ratio of *G* = *u*g/*u**∞* = 2 (where *u*g and *u**∞* are the gust and freestream velocities, respectively). In this 3D environment, rapid variations in the effective angle of attack cause severe load fluctuations and complex vortex–gust interactions. The RL agent uses three leading-edge jet actuators to directly inject momentum into the separated boundary layer. This actuation creates a more compact, energetic separation bubble and generates primary vortices with enhanced circulation strength and spatial coherence. By synchronizing the controlled vortex shedding with the natural flow response to the incoming gust, this dynamic circulation control mechanism prevents premature vortex breakdown and destructive interference. The resulting wake has a reduced velocity deficit and decreased turbulence intensity, allowing the airfoil to maintain stable aerodynamic performance with approximately 20% lower load oscillations despite extreme transverse perturbations (Fig. [2](/articles/s41586-026-10917-6#Fig2)). ## Scaling complexity in HydroGym Although conventional reinforcement learning has proven effective for flow control, emerging RL paradigms offer fundamentally different approaches to optimization and learning efficiency. HydroGym’s advanced capabilities demonstrate how differentiable reinforcement learning exploits gradient information through entire simulation trajectories, enabling more efficient policy optimization, whereas multi-agent frameworks address the spatial complexity of three-dimensional flows through distributed actuator coordination. Furthermore, transfer learning capabilities enable knowledge reuse across Reynolds numbers, geometries and dimensions, substantially reducing computational requirements for new control problems. Together, these approaches provide direct access to sensitivity information, enable scalable solutions for complex 3D control scenarios and unlock generalizable control strategies. ### Gradient-enhanced policy optimization through differentiable physics In contrast to traditional CFD solvers, modern machine learning libraries enable physics solvers to compute sensitivities of objective functions with respect to control parameters and actuator placement by leveraging automatic differentiation. This capability allows policy optimization using sensitivity information, yielding improved performance and sample efficiency[28](/articles/s41586-026-10917-6#ref-CR28),[29](/articles/s41586-026-10917-6#ref-CR29). HydroGym provides a variety of differentiable environments, including 2D Kolmogorov flows, 3D turbulent channel flows, and shock-vector-control scenarios for flows in 2D and 3D nozzles that feature complex shock interactions and system dynamics. Here we briefly demonstrate the advantages of differentiable environments for the chaotic Kolmogorov flow[30](/articles/s41586-026-10917-6#ref-CR30) characterized by extreme energy dissipation events from nonlinear energy transfer across scales. The control task is to leverage these events to enhance mixing through adaptive forcing wavenumbers. We compared standard PPO against gradient-enhanced PPO (GPPO), where agents controlled the amplitudes of the forcing terms. In GPPO, the loss function backpropagates through both the policy and environment, via an analytical gradient term ([Supplementary Information](/articles/s41586-026-10917-6#Sec40)). Gradient-enhanced PPO reduced training iterations by at least 65% while producing more efficient controllers with lower action amplitudes than PPO (Fig. [3](/articles/s41586-026-10917-6#Fig3)). These results demonstrate that incorporating gradient information into the optimization process substantially improves sample efficiency and yields more efficient policies. As differentiable simulation and control methods mature, more sophisticated gradient-based approaches may deliver further gains in both efficiency and performance. ### Multi-agent coordination for spatially distributed 3D cylinder control The HydroGym benchmark platform further supports multi-agent reinforcement learning (MARL) in a variety of environments. For a 3D cylinder at Re = 3,900, for instance, the MARL framework exploits the inherent spatial locality of flow control by decomposing the global control problem into multiple local pseudo-environments[31](/articles/s41586-026-10917-6#ref-CR31), each managing a pair of ZNMF jets distributed spanwise along the cylinder surface. Unlike the work by Suárez and colleagues[31](/articles/s41586-026-10917-6#ref-CR31), our approach uses shared gradient updates and a unified replay buffer across all agents, enhancing computational efficiency while maintaining the cooperative learning dynamics essential for coordinated actuation. As shown in Fig. [3](/articles/s41586-026-10917-6#Fig3), this architecture enables the discovery of efficient control strategies that use diverse frequency bandwidths and adaptive spanwise coordination, resulting in drag reductions of approximately 8%, similar to those in the work by Suárez and co-workers[31](/articles/s41586-026-10917-6#ref-CR31). The agents learn to manipulate the shear layer instabilities, effectively enlarging the recirculation bubble and reshaping the wake into a more aerodynamically efficient teardrop-like configuration. The cooperative control strategy attenuates Reynolds stress fluctuations while shifting peak turbulent activity downstream, resulting in reduced pressure drag through pressure recovery at the rear surface. This distributed actuation offers a scalable path for active flow control in complex 3D flows while maintaining computational tractability by exploiting local invariances in the flow. The shared learning architecture allows agents to benefit from collective experience, which leads to more robust and generalizable control policies. Similar MARL successes in robotic swarm coordination, autonomous vehicle networks and multi-robot manipulation demonstrate the broad applicability of this approach for distributed control. ### Transfer learning capabilities A fundamental question motivating the core HydroGym idea is whether discovered mechanisms represent configuration-specific solutions or more generalizable approaches. To address this, we systematically investigate how control knowledge transfers across Reynolds numbers, geometries and dimensions, showing that HydroGym enables efficient transfer learning to reduce computational requirements and improve controller generalization. When transferring control policies across a wide range of Reynolds numbers, fine-tuning achieves optimal performance in approximately 50% fewer episodes than training from scratch (Extended Data Fig. [3](/articles/s41586-026-10917-6#Fig7), rows 1 and 2). Drag reduction effectiveness is either maintained or improved throughout this range, indicating that flow control strategies capture scale-invariant physics—particularly pressure gradient manipulation and the timing of momentum injection relative to vortex formation cycles. Beyond Reynolds number invariance, the policies demonstrate geometric transferability. Control strategies for vortex shedding suppression learned on circular cylinders effectively manage vortex shedding in square geometries. This supports the notion that policies learn mathematical structures governing pattern formation and symmetry breaking that persist across boundary conditions, rather than geometry-specific control (Extended Data Fig. [3](/articles/s41586-026-10917-6#Fig7), row 3). The most stringent test is dimensional transfer from 2D to 3D, which confirms universality across spatial scales. Despite the increased complexity of three-dimensional flows, including spanwise coupling effects and further instability modes, pre-trained 2D control strategies successfully delay separation and maintain effective drag reduction in 3D applications while reducing computational costs by about 50−60% (Fig. [3](/articles/s41586-026-10917-6#Fig3) and Extended Data Fig. [3](/articles/s41586-026-10917-6#Fig7), row 4). This indicates that boundary layer manipulation strategies transcend spatial dimensionality through conserved scaling laws and similarity parameters, with 2D training capturing the dominant flow physics relevant for 3D control. The transfer mechanism itself provides insights into learning. Although zero-shot performance varies across scenarios, the subsequent fine-tuning process consistently benefits from pre-trained initialization, even when immediate transfer seems limited. This pattern suggests that learned policies encode fundamental flow control relationships that may not manifest immediately in performance metrics but provide essential structure for accelerated learning, indicating that deep RL discovers mathematical invariants governing scaled flow problems. ## Physics-guided zero-shot transfer to a turbulent airfoil wing Building on these demonstrated spatial and geometric transfer capabilities, we next push the limits of generalizability to a much more challenging problem: transferring policies to industrially relevant, high-Reynolds-number environments. Directly training reinforcement learning policies on complex, three-dimensional wings at high Reynolds numbers is computationally prohibitive. To overcome this, we hypothesize that computationally tractable canonical environments can serve as surrogates for discovering transferable control priors, breaking the specificity trap that currently dominates flow control research. To demonstrate this hypothesis, we use a surrogate training paradigm that exploits partial invariances in wall-bounded turbulence. Specifically, we train multi-agent RL controllers exclusively on a canonical HydroGym turbulent channel flow at Re*τ* = 206. Although channel flows lack the streamwise-varying pressure gradients, surface curvature and inhomogeneous boundary layer development of airfoil flows, they reproduce the inner-layer turbulence scaling and streak dynamics that dominate near-wall momentum transport. Within these channel surrogates, controllers learn distributed actuation strategies that target near-wall flow structures via high-speed streak attenuation and coherent vortex manipulation. After training, we then deploy these channel-trained controllers zero-shot to a three-dimensional NACA0012 wing section at Re*c* = 200,000. Despite the radical differences in the geometry, pressure gradient and Reynolds number, the results are striking. The learned mechanisms demonstrably encode relationships that remain effective across training-deployment mismatch, achieving a 38% reduction in local skin-friction drag (Fig. [4](/articles/s41586-026-10917-6#Fig4)). Importantly, this zero-shot result substantially outperforms current gold-standard controllers, such as opposition control. We interpret this as a proof of concept for transferable control strategies: control priors discovered in a simple environment can remain effective on a substantially more complex aerodynamic configuration that shares the same near-wall turbulence physics. The extent to which such transfer generalizes across configurations that do not share these features—for example, flows dominated by strong pressure gradients, separation or compressibility—remains an open question that HydroGym is designed to investigate systematically. The computational advantage is substantial: channel surrogate training requires two-orders-of-magnitude fewer grid points than wing simulations, while the additional homogeneous streamwise direction permits aggressive parallelization. Accounting for both spatial resolution and temporal dynamics, this yields an exploration efficiency gain that exceeds 104 in computational time. Even with periodic validation on the full wing geometry, total wall-clock time is reduced by approximately a factor of 60 relative to direct on-wing training. This efficiency enables systematic policy optimization including hyperparameter searches, multi-agent coordination strategies and actuation pattern exploration that would be prohibitively expensive at wing-scale Reynolds numbers. ## Discussion HydroGym addresses the fundamental bottlenecks preventing reinforcement learning from transforming fluid dynamics control: the computational intractability of training effective policies on expensive CFD simulations, and the considerable domain expertise required to implement these policies and simulations. Our platform’s solver-independent architecture and comprehensive benchmark suite demonstrate that standardized environments can accelerate algorithmic development in fluid control just as ImageNet[9](/articles/s41586-026-10917-6#ref-CR9) transformed computer vision and MuJoCo[10](/articles/s41586-026-10917-6#ref-CR10) revolutionized robotics research. HydroGym provides an extensive suite of rigorously validated flow environments with comprehensive RL benchmarks. These environments include 2D and 3D flows across a range of geometries and Reynolds numbers, with a diversity of control challenges. Furthermore, the HydroGym framework supports advanced RL features, including differentiable environments, transfer learning and multi-agent RL. Our physics-guided zero-shot transfer is a proof of concept for transferable control strategies. By deploying controllers trained on a simplified surrogate environment to a complex, three-dimensional wing section, we show that deep reinforcement learning can learn control strategies that transfer, at least between configurations sharing near-wall turbulence physics, rather than overfitting to a specific geometry. This surrogate approach bypasses the computational bottlenecks of direct high-Reynolds-number training, unlocking control discovery for industrial scales that previously exceeded computational feasibility. Whether this transferability extends across the broader diversity of fluid-mechanical systems remains an open question, and mapping those limits is an important direction that HydroGym is built to support. Moving forward, extending this paradigm to span varying Reynolds numbers, pressure gradients, and geometric complexities will map the boundaries of transferability and accelerate development—from turbomachinery to vehicle aerodynamics. This exemplifies HydroGym’s core mission: moving the field from isolated case studies toward accumulating, reproducible progress on flow control as a community challenge. ### Limitations and future directions Although HydroGym addresses fundamental barriers in applying reinforcement learning in fluid control, several current limitations define the present scope and motivate ongoing developments. #### Data sources and physical assumptions All current HydroGym environments employ direct numerical simulation (DNS) with fully resolved turbulent structures, providing validated, benchmarked ground truth without turbulence modelling assumptions or closure relations. This approach ensures that learned control strategies exploit genuine physical mechanisms rather than artifacts of turbulence models. However, DNS is computationally intensive, particularly at higher Reynolds numbers. The platform currently spans Reynolds numbers up to ${{\rm{Re}}}_{\tau }=2,200$ for internal flows and ${{\rm{Re}}}_{c}=400,000$ for external aerodynamics—regimes sufficient for fundamental research and algorithm development, but below the Re ≈ 107–108 characteristic of many industrial applications. Critically, this limitation reflects training-phase computational constraints rather than solver capabilities; our production solvers routinely handle ${\mathcal{O}}(1{0}^{9})$ degree-of-freedom simulations on full aircraft geometries. However, embedding such simulations in RL training loops would require enormous compute resources (requiring more than 500 GPUs and over 50 TB of memory per training run), rendering systematic algorithmic development impractical. Importantly, our demonstrated physics-guided zero-shot transfer validates that computationally tractable canonical environments can serve as surrogates for discovering reusable control priors applicable to industrially relevant configurations, potentially bridging the Reynolds number gap. #### Flow regime coverage Current environments emphasize canonical flows and near-equilibrium turbulent boundary layers, reflecting established validation benchmarks. Non-equilibrium flows characterized by strong pressure gradients, flow separation, reattachment and transient dynamics present further complexity. We are actively developing environments that incorporate adverse and favourable pressure gradients using body forces. Moreover, future environment releases will include an extended range for multi-physics coupling, including aero-acoustic scenarios (jet noise mitigation, trailing-edge noise suppression), reactive flows in combustion systems and multiphase dynamics. #### Sample efficiency and reduced-order models Despite advances through differentiable environments and multi-agent architectures, effective policy learning still requires thousands of environment interactions, creating computational bottlenecks for complex 3D turbulent flows. Although our differentiable environments accelerate learning using gradient information, highly turbulent 3D flows still face fundamental numerical challenges in maintaining stable automatic differentiation through chaotic dynamics. Reduced-order models offer a promising avenue for dramatic efficiency gains[32](#ref-CR32),[33](#ref-CR33),[34](#ref-CR34),[35](/articles/s41586-026-10917-6#ref-CR35). Recent work demonstrating that sparse learning methods can approximate flow dynamics and reward functions with up to 100× fewer high-fidelity interactions[21](/articles/s41586-026-10917-6#ref-CR21) suggests a hybrid training paradigm: construct fast surrogate environments from limited DNS data, perform extensive policy optimization in the surrogate, then transfer and refine on high-fidelity simulations. Reduced-order model integration introduces further considerations, particularly regarding energy exchange between truncated and retained modes in non-equilibrium regimes, which must be carefully addressed in future implementations. #### Data accessibility and storage constraints The platform integrates trajectory and flow field uploads to Hugging Face repositories, enabling offline reinforcement learning, surrogate model training and community data sharing. However, data volumes pose practical limits: for our largest simulations with ${\mathcal{O}}(1{0}^{9})$ degrees of freedom, individual flow fields require 75 GB of storage, which means that a single 200-step test trajectory generates 15 TB of data. This precludes comprehensive training dataset dissemination but enables selective high-value data sharing and compressed state representations. #### Algorithmic development priorities Beyond expanding environment coverage, several algorithmic directions promise future impact. Deep coupling between RL agents and turbulence modelling—where agents learn closure parameters for Reynolds-averaged Navier–Stokes equations, subgrid-scale models or wall functions on-the-fly during simulation[20](/articles/s41586-026-10917-6#ref-CR20)—could extend the scope. Advancing differentiable physics capabilities to handle fully turbulent flows would dramatically improve sample efficiency and enable gradient-based optimization at currently inaccessible scales[36](/articles/s41586-026-10917-6#ref-CR36),[37](/articles/s41586-026-10917-6#ref-CR37). Surrogate models with uncertainty estimates may guide strategic high-fidelity sampling. Incorporating causal learning could enhance policy robustness and interpretability by identifying invariant physical mechanisms[38](#ref-CR38),[39](#ref-CR39),[40](#ref-CR40),[41](#ref-CR41),[42](#ref-CR42),[43](/articles/s41586-026-10917-6#ref-CR43), transforming black-box controllers into interpretable cause-and-effect chains. Hierarchical multi-agent architectures may enable control across multiple spatial and temporal scales, from local boundary layer manipulation to global patterns. Overall, the results presented here point towards a broader architectural vision: a foundation model for flow control, pretrained across the diversity of HydroGym environments, and capable of zero-shot or few-shot adaptation to new configurations. The analogy to large language models is instructive as their generalization emerges not from any single algorithmic breakthrough, but from the availability of a standardized, large-scale and diverse pretraining corpus. Fluid flow control has lacked an equivalent substrate: a unified collection of physically validated, computationally tractable flow control environments spanning geometries, Reynolds numbers and flow regimes, with consistent interfaces for policy training and evaluation. HydroGym is designed to be precisely that substrate. The transferability results we demonstrate are a strong indicator that such pretraining is meaningful, where control priors learned in canonical flows encode reusable physical structure rather than environment-specific artifacts. Realizing this vision will require systematically mapping the boundaries of transferability across flow families, developing architectures that condition on physical context, and curating pretraining corpora that balance diversity with physical coherence. Eventually, HydroGym agents will be deployed in experimental flow control scenarios, further extending and testing their limits. These are tractable research questions, and HydroGym provides the benchmarks and infrastructure to pursue them rigorously. ## Methods The HydroGym platform formulates flow control problems as discrete-time Markov decision processes, defined by the tuple $({\mathcal{S}},{\mathcal{A}},{\mathcal{P}},{\mathcal{R}})$. At each control step *t*, the agent: receives an observation state ${s}_{t}\in {\mathcal{S}}$, which represents flow measurements; executes an action ${a}_{t}\in {\mathcal{A}}$, which represents actuator inputs; and receives a scalar reward ${r}_{t}\in {\mathcal{R}}$ that quantifies the control objective (for example, drag reduction or flow stabilization). The transition probability ${\mathcal{P}}$ is governed implicitly by the underlying Navier–Stokes equations integrated by the fluid solver. To bridge the gap between machine learning and fluid dynamics, HydroGym uses the Farama Foundation Gymnasium interface[44](/articles/s41586-026-10917-6#ref-CR44), abstracting the complexity of the CFD execution. This abstraction is illustrated by the straightforward initialization and interaction pattern for an open cavity control example: # Swap one line to switch backend solvers import hydrogym.firedrake as hgym # import hydrogym.maia as hgym # import hydrogym.nek as hgym # import hydrogym.jax as hgym # import hydrogym.jaxfluids as hgym # initialize environment env = hgym.FlowEnv('Cavity_2D_Re7500', **kwargs) # reset environment obs, info = env.reset() # interact with environment for i in range(num_interactions): obs, reward, terminated, truncated, info = env.step(action) This design enables control and machine learning engineers to focus on algorithmic development without CFD expertise, while providing fluids researchers access to state-of-the-art RL techniques. The solver-independent architecture ensures that advances in CFD can be seamlessly integrated into the platform without disrupting existing workflows. A critical element of the Markov decision processes formulation in HydroGym is the temporal coupling between the physical solver and the RL agent. As turbulent flows evolve continuously while RL agents typically assume discrete actions, control inputs are updated at fixed intervals *Δ**t*ctrl, which correspond to multiple physical CFD time steps designed to capture the characteristic timescale of the target instabilities (for example, vortex shedding or shear-layer oscillation). To prevent numerical instabilities arising from discontinuous boundary conditions, HydroGym enforces temporal smoothing on all control actions. The instantaneous actuation magnitude *A*(*t*) is smoothly interpolated between the previous action *A*old and the new target action *A*new over a predefined number of sub-iterations using a hyperbolic tangent or exponential ramp function. Furthermore, to standardize learning across disparate physical regimes, all observation features and action bounds are inherently normalized to the [−1, 1] range within the environment wrapper. ### Computational fluid dynamics backends To accommodate diverse computational scales ranging from rapid prototyping to large-scale high-performance computing (HPC) campaigns, HydroGym implements a solver-independent architecture supporting multiple backends. All computational environments have been thoroughly validated against established benchmarks to ensure physical correctness and high-fidelity resolution of the underlying fluid dynamics. Comprehensive validation studies and grid convergence analyses for each solver and flow configuration are detailed in Supplementary Sections [3](/articles/s41586-026-10917-6#MOESM1) and [5](/articles/s41586-026-10917-6#MOESM1). #### Lattice Boltzmann solver (m-AIA LB) For the majority of weakly compressible, large-scale two- and three-dimensional DNS, HydroGym uses the lattice Boltzmann method embedded within the m-AIA solver framework[45](/articles/s41586-026-10917-6#ref-CR45), which has been continuously developed at RWTH Aachen University for over two decades. The solver evolves the discrete particle distribution function *f**i* on a D3Q27 velocity lattice via a two-step collision–streaming update. At low-to-moderate Reynolds numbers, a Bhatnagar–Gross–Krook (BGK) collision operator is used, relaxing distributions towards a local Maxwellian equilibrium with a single relaxation frequency *ω*BGK linked to the kinematic viscosity. To enhance stability at higher Reynolds numbers, a cumulant-based collision operator is used instead, performing relaxations in a cumulant space with individual rates *ω**α* (ref. [46](/articles/s41586-026-10917-6#ref-CR46)). Local grid refinement follows the methodology proposed by Eitel-Amor and colleagues[47](/articles/s41586-026-10917-6#ref-CR47), adjusting *ω*BGK across hierarchical grid levels to preserve a constant kinematic viscosity throughout the domain. No-slip boundaries are represented via a second-order interpolated bounce-back scheme. A hybrid parallelization strategy built on the message passing interface (MPI) and a shared memory model either based on open multi-processing (OpenMP) or on the parallel algorithms in the C++ standard provided by the NVIDIA HPC SDK and AMD ROCm HIPSTDPAR backends allows for a hardware-agnostic GPU implementation. #### Finite-volume solver (m-AIA FV) Beyond the lattice Boltzmann method, m-AIA includes a compressible Navier–Stokes solver based on a finite-volume method targeting DNS and large-eddy simulation (LES) of wall-bounded turbulent flows at higher Mach numbers. The compressible Navier–Stokes equations are cast in an arbitrary Lagrangian–Eulerian formulation and discretized on structured, body-conforming curvilinear grids with a cell-centred scheme, which enables anisotropic mesh refinement in the wall-normal direction suited for flat-plate and airfoil configurations. Inviscid fluxes are evaluated with the advection upstream splitting method combined with Monotonic Upstream-centred Scheme for Conservation Laws reconstruction for second-order spatial accuracy, whereas viscous fluxes are discretized using a modified cell-vertex approach. Temporal integration uses a five-stage, low-storage Runge–Kutta scheme; when the grid deforms under surface actuation, the geometric conservation law is enforced at every stage. Subgrid-scale effects in under-resolved LES are handled implicitly via a monotonically integrated LES strategy, exploiting the inherent numerical dissipation of the upwind-biased advection upstream splitting method scheme. The finite-volume code shares the same MPI/OpenMP/GPU parallelization strategy as the lattice Boltzmann solver. For RL integration, the solver exposes both synthetic jet actuation and a travelling transversal surface-wave boundary condition[14](/articles/s41586-026-10917-6#ref-CR14),[48](#ref-CR48),[49](#ref-CR49),[50](#ref-CR50),[51](/articles/s41586-026-10917-6#ref-CR51), where the wave amplitude *A*, wavelength *λ*, and phase speed *c* serve as time-varying control parameters updated by the agent at each control step via a *C*1-continuous cosine cross-fade transition to ensure numerical stability. #### Spectral-element solver (Nek5000) For incompressible high-Reynolds-number configurations requiring higher-order accuracy, HydroGym provides a Nek5000[52](/articles/s41586-026-10917-6#ref-CR52) backend using the spectral-element method. The velocity and pressure fields are approximated by high-order Lagrange interpolants on hexahedral elements following the ${{\mathbb{P}}}_{N}{{\mathbb{P}}}_{N-2}$ formulation[53](/articles/s41586-026-10917-6#ref-CR53): velocity is collocated onto *N*3 Gauss–Lobatto–Legendre points per element, whereas pressure resides on a staggered grid of (*N* − 2)3 Gauss–Legendre points. Temporal integration combines a third-order explicit extrapolation scheme for the nonlinear convective terms with a third-order implicit backward differentiation scheme for viscous contributions. Aliasing errors are mitigated by overintegration, with an oversampling factor of 3/2 in each spatial direction. Flow actuation is prescribed as a time-dependent wall-normal Dirichlet velocity *v**n* with the spatial mean removed to satisfy the ZNMF constraint. Parallelization is achieved through a hybrid OpenMPI distributed- and shared-memory strategy, and coupling with RL agents is implemented via dynamic MPI-based communication. #### Differentiable incompressible solver To enable gradient-enhanced reinforcement learning[28](/articles/s41586-026-10917-6#ref-CR28), HydroGym provides natively differentiable solvers written in JAX[54](/articles/s41586-026-10917-6#ref-CR54) for the incompressible Navier–Stokes equations. The Kolmogorov flow environment uses a pseudo-spectral method with two-thirds de-aliasing on a doubly periodic domain, advancing the vorticity–streamfunction formulation with a fourth-order Runge–Kutta scheme. The three-dimensional turbulent channel flow uses a fully differentiable finite-difference DNS solver, in which spatial derivatives are formed with sparse differentiation matrices, and the pressure Poisson equation is solved via a biconjugate gradient-stabilized method. Both environments implement a Gymnax interface[55](/articles/s41586-026-10917-6#ref-CR55) and inherit RL frameworks from PureJaxRL[56](/articles/s41586-026-10917-6#ref-CR56), enabling a fully synchronous training pipeline compatible with JAX’s automatic differentiation. Exact analytical gradients of the objective with respect to control parameters are obtained via jax.grad through the entire simulation trajectory. Parallelization across multiple random seeds is achieved using vmap for batch processing, and just-in-time compilation via jit optimizes execution on GPU hardware. #### Differentiable compressible solver (JAX-Fluids) For gradient-based flow control in compressible and multiphase regimes, HydroGym integrates the JAX-Fluids solver[57](/articles/s41586-026-10917-6#ref-CR57),[58](/articles/s41586-026-10917-6#ref-CR58)—a fully-differentiable, high-order CFD code written entirely in Python using the JAX library. The compressible Navier–Stokes equations are solved on structured Cartesian grids, with arbitrary one-dimensional mesh stretching. Convective fluxes are evaluated using either fifth-order weighted essentially non-oscillatory (WENO5-Z), or sixth-order targeted essentially non-oscillatory (TENO6-A) shock-capturing reconstruction, coupled with an approximate Harten–Lax–van Leer–Contact Riemann solver. Diffusive fluxes are evaluated using high-order central differences, and time integration is performed with explicit total variation diminishing Runge–Kutta schemes. Multiphase flows are supported through both a sharp-interface level-set method and a five-equation diffuse-interface model. Positivity-preserving limiters are used to ensure robust integration in the presence of strong shocks or large density ratios. The level-set implementation in JAX-Fluids further functions as an immersed boundary method, enabling flow simulations around complex geometries. The immersed boundary method is particularly well suited for active flow control problems, as control actions (for example, blowing or suction) can be naturally incorporated through interface exchange terms. By adhering strictly to JAX’s functional programming model, the entire simulation pipeline is end-to-end differentiable. Exact gradients of arbitrary scalar objectives, for example, time-averaged drag or RL reward, with respect to control inputs or neural network weights are backpropagated through the full temporal trajectory via jax.grad or value_and_grad, with gradient checkpointing (jax.checkpoint) mitigating memory overhead for long rollouts. For large-scale HPC deployment, JAX-Fluids uses homogeneous domain decomposition across multiple accelerated linear algebra devices via jax.pmap, with inter-block halo exchanges performed exclusively through jax.lax.ppermute to keep the automatic differentiation graph intact across distributed, multi-node clusters. #### Finite-element solver (Firedrake) For maximum code transparency and rapid prototyping of two-dimensional control problems, HydroGym provides a Firedrake[59](/articles/s41586-026-10917-6#ref-CR59) backend built on the Portable, Extensible Toolkit for Scientific Computation, which offers automatic code generation for variational problems. The incompressible Navier–Stokes equations are discretized with Taylor–Hood elements (second-order continuous Galerkin for velocity, first-order for pressure) to ensure inf–sup stability, and time integration uses fully implicit schemes, with an automatic solver-parameter selection based on the Reynolds number and grid resolution. The implementation follows a three-tier modular architecture separating the physical problem definition (PDEBase), time-stepping (TransientSolver) and RL interfacing (FlowEnv), with the last module implementing the Farama Foundation Gymnasium interface and translating between RL concepts and CFD operations. Parallelization leverages Firedrake’s distributed mesh capabilities and the Portable, Extensible Toolkit for Scientific Computation's parallel sparse linear algebra routines. #### Reinforcement learning and multi-agent infrastructure Rather than heavily tuning hyperparameters for each configuration, which would hinder generalizability, HydroGym relies on robust observation and action normalization to ensure standard off-the-shelf RL algorithms serve as strong baselines. The platform integrates seamlessly with StableBaselines3[60](/articles/s41586-026-10917-6#ref-CR60), TorchRL[61](/articles/s41586-026-10917-6#ref-CR61) and CleanRL[62](/articles/s41586-026-10917-6#ref-CR62) for model-free continuous control, and PureJaxRL for natively differentiable environments. For standard model-free environments, we evaluated PPO, DDPG and TD3. #### Gradient-enhanced PPO HydroGym implements GPPO for differentiable environments. Standard PPO relies on likelihood ratio methods or generalized advantage estimation to estimate policy gradients. In GPPO, the analytical gradient of the reward trajectory with respect to the policy parameters, ${\nabla }_{\theta }{{\mathbb{E}}}_{\tau \sim {\pi }_{\theta }}[\sum R({s}_{t},{a}_{t})]$ is computed exactly by backpropagating through the deterministic fluid dynamics solver. This analytical gradient is incorporated into PPO’s clipped surrogate objective, drastically reducing variance and improving sample efficiency. #### Multi-agent reinforcement learning HydroGym implements a decentralized MARL architecture to tackle the dimensionality barrier of spatially distributed 3D flows (such as 3D cylinders, channel and airfoil flows). Global control domains are partitioned into locally invariant pseudo-environments. Multiple agents operate on these local partitions and may share a common control policy π(*a*∣*s*). To ensure unbiased data distributions in the shared replay buffer, overlapping mesh nodes are excluded from the pseudo-environment definitions, whereas identical actuation is enforced at boundary interfaces to preserve control continuity. Communication between the parallelized CFD domains and the centralized RL policy is managed efficiently via an MPI-based interface. Further details regarding the reinforcement learning agents are provided in Supplementary Section [4](/articles/s41586-026-10917-6#MOESM1). ### Environment set-up for discussed results Although the configurations discussed in the main text are briefly outlined below, comprehensive details concerning the physical characteristics, exact boundary conditions, observation or action space normalizations, and specific reward formulations for all environments are provided in Supplementary Section [5](/articles/s41586-026-10917-6#MOESM1). #### Circular cylinder (Re = 3,900) The subcritical 3D cylinder flow represents a highly chaotic wake. The computational domains extend [51.2*D* × 48*D*] in 2D and 32*D* × 16*D* × 4*D* with periodic spanwise boundaries in 3D. Actuation is applied via ZNMF synthetic jets positioned at the top and bottom of the cylinder. The reward function targets drag minimization while penalizing lift oscillations: *r* = −∣*C**D*∣ − *ω*∣*C**L*∣. For MARL configurations, the span is decomposed into independent pseudo-environments, each controlling local jet pairs. #### Fluidic pinball (Re = 100−150) The pinball environment features three circular cylinders in an equilateral triangle layout, exposing agents to multiple bifurcation regimes, including symmetry-breaking pitchfork bifurcations and chaotic dynamics. The control mechanism consists of the independent surface rotation of all three cylinders. The reward formulation targets collective drag reduction: $r=-{\sum }_{i=1}^{3}| {C}_{D,i}| \,-$ $\omega {\sum }_{i=1}^{3}| {C}_{L,i}| $, where the scaling factor *ω* restricts policies from exploiting asymmetric lift generation. #### Open cavity flow (Re = 4,200−7,500) The cavity flow targets the stabilization of complex feedback loops and Kelvin–Helmholtz instabilities at the shear layer spanning the cavity opening. The reward function penalizes deviations of observed flow quantities from a target reference state: $r=-{\sum }_{i}{\left(\frac{{{\rm{obs}}}_{i}-{\bar{o}}_{i}}{{{\sigma }}_{i}}\right)}^{2}$, where obs*i* represents pressure and velocity measurements at shear-layer probes, ${\bar{o}}_{i}$ is the target mean approximated over 1,000 instability cycles and *σ**i* provides normalization scaling. Actuation is performed using localized jet actuators at the upstream cavity edge and, optionally, inside the cavity, providing multi-point flow manipulation to disrupt resonant interactions. #### Transverse gust mitigation (NACA0012, Re = 1,000) This environment simulates extreme aerodynamic conditions using a highly disturbed inflow. A 1-cosine transverse gust with a gust ratio *G* = 2.0 interacts with an airfoil at a high angle of attack (*α* = 20°), threatening dynamic stall and severe load fluctuations. Control is achieved via three independent jet actuators distributed along the leading edge, each covering 3% of the chord length. The reward function is designed to minimize gust-induced force variance while preserving the baseline aerodynamic efficiency: $r=-| {C}_{L}(t)-{\bar{C}}_{L,\mathrm{ref}}| -\omega | {C}_{D}(t)-{\bar{C}}_{D,\mathrm{ref}}| $. ### Physics-guided transfer learning and zero-shot deployment HydroGym uses a physics-guided zero-shot transfer protocol to bridge the gap between computationally tractable RL training and prohibitive industrial-scale CFD. This method was used to control the suction-side turbulent boundary layer (TBL) of a NACA0012 wing section at a chord-based Reynolds number of Rec = 200,000. #### Surrogate construction and pre-training Direct RL exploration on the high-resolution wing is computationally prohibitive. Instead, our working hypothesis argues that the target control region on the wing (ranging from *x*/*c* = 0.25 to 0.86) can be systematically partitioned into smaller chordwise blocks based on the local Clauser pressure-gradient parameter *β*, with a surrogate environment constructed for each. For the symmetric NACA0012 configuration at a 0° angle of attack evaluated here, the moderate adverse pressure gradient across the control region naturally yields a single, continuous control block. Consequently, a single surrogate turbulent channel flow (TCF) is generated. Furthermore, the near-wall spatial resolution of the TCF matches the wing simulations in viscous units (*Δ**x*+, *Δ**y*+, *Δ**z*+), ensuring that the effective footprint of the controllers remains geometrically consistent. Agents are trained exclusively within these TCF surrogates using the MARL framework (TD3 algorithm). The observation space relies strictly on near-wall metrics accessible in real-world scenarios: the wall-tangential ($u{\prime} $) and wall-normal ($v{\prime} $) velocity fluctuations sampled at a sensing plane of *y*+ = 15. The action *a* dictates the wall-normal blowing or suction velocity, strictly bounded by the local friction velocity (−*u**τ* ≤ *a* ≤ *u**τ*) and subjected to a ZNMF constraint across the control region. The reward function seeks to minimize the relative wall-shear stress, $r=1-{\tau }_{w}^{\mathrm{ctrl}}/{\tau }_{w}^{\mathrm{ref}}$. #### Zero-shot deployment Following training in the idealized TCF surrogates, the optimized policies are deployed directly onto the corresponding partitioned blocks of the 3D NACA0012 wing sections. No further on-wing training or fine-tuning is performed. To map the policies appropriately, the agent’s observations are standardized using the spatially varying viscous scales of the wing. As the viscous time unit *t** evolves along the wing chord, the update frequency of the deployed actions is scaled by the local streamwise-averaged friction velocity ${\langle {u}_{\tau }\rangle }_{x}$ of each respective block. By learning generalized responses to near-wall streaks rather than overfitting to the macroscopic geometry of the channel, the RL policies successfully attenuate high-speed streaks on the wing, yielding substantial reductions in skin-friction drag at a fraction of the computational cost (~104 magnitude reduction in required core-hours compared to direct on-wing training). Baseline comparison methods include uniform blowing and opposition control[63](/articles/s41586-026-10917-6#ref-CR63). ## Data availability The flow-field trajectories and datasets underlying the benchmark results reported in this work are available through the HydroGym Hugging Face repositories at [https://huggingface.co/datasets/dynamicslab/HydroGym-environments](https://huggingface.co/datasets/dynamicslab/HydroGym-environments). Owing to their volume, the largest direct-numerical-simulation datasets (individual flow fields up to 75 GB; full test trajectories up to 15 TB) cannot be deposited in their entirety; representative and compressed subsets sufficient to reproduce the reported results are provided, and complete datasets are available from the corresponding authors on reasonable request. Source data are provided with this paper. ## Code availability The HydroGym platform is publicly available at [https://dynamicslab.github.io/hydrogym/](https://dynamicslab.github.io/hydrogym/). A versioned snapshot of the code used in this study is archived on Zenodo at [https://doi.org/10.5281/zenodo.21062222](https://doi.org/10.5281/zenodo.21062222) (ref. [64](/articles/s41586-026-10917-6#ref-CR64)). Pre-built docker containers are provided at [https://hub.docker.com/repositories/clagemann](https://hub.docker.com/repositories/clagemann). ## References 1. Marusic, I. et al. An energy-efficient pathway to turbulent drag reduction. *Nat. Commun.***12** , 5805 (2021). 2. Font, B., Alcántara-Ávila, F., Rabault, J., Vinuesa, R. & Lehmkuhl, O. Deep reinforcement learning for active flow control in a turbulent separation bubble. *Nat. Commun.***16** , 1422 (2025). 3. Lagemann, E., Brunton, S., Schröder, W. & Lagemann, C. Towards extending the aircraft flight envelope by mitigating transonic airfoil buffet. *Nat. Commun.***15** , 5020 (2024). 4. Brunton, S. & Noack, B. R. Closed-loop turbulence control: progress and challenges. *Appl. Mech. Rev.***67** , 050801 (2015). 5. Brunton, S., Noack, B. R. & Koumoutsakos, P. Machine learning for fluid mechanics. *Ann. Rev. Fluid Mech.***52** , 477–508 (2020). 6. Koumoutsakos, P. On roads less travelled between AI and computational science. *Nat. Rev. Phy.***6** , 342–344 (2024). 7. Jumper, J. et al. Highly accurate protein structure prediction with AlphaFold. *Nature***596** , 583–589 (2021). 8. Degrave, J. et al. Magnetic control of tokamak plasmas through deep reinforcement learning. *Nature***602** , 414–419 (2022). 9. Deng, J. et al. ImageNet: a large-scale hierarchical image database. In *2009 IEEE Conference on Computer Vision and Pattern Recognition* 248–255 (IEEE, 2009). 10. Todorov, E., Erez, T. & Tassa, Y. Mujoco: a physics engine for model-based control. In *2012 IEEE/RSJ International Conference on Intelligent Robots and Systems* 5026–5033 (IEEE, 2012). 11. Kochkov, D. et al. Machine learning–accelerated computational fluid dynamics. *Proc. Natl Acad. Sci. USA***118** , e2101784118 (2021). 12. Suárez, P. et al. Flow control of three-dimensional cylinders transitioning to turbulence via multi-agent reinforcement learning. *Commun. Eng.***4** , 113 (2025). 13. Chatzimanolakis, M., Weber, P. & Koumoutsakos, P. Learning in two dimensions and controlling in three: generalizable drag reduction strategies for flows past circular cylinders through deep reinforcement learning. *Phys. Rev. Fluids***9** , 043902 (2024). 14. Mäteling, E., Albers, M. & Schröder, W. How spanwise travelling transversal surface waves change the near-wall flow. *J. Fluid Mech.***957** , A30 (2023). 15. Ricco, P., Skote, M. & Leschziner, M. A. A review of turbulent skin-friction drag reduction by near-wall transverse forcing. *Prog. Aerosp. Sci.***123** , 100713 (2021). 16. Zhao, H., Zhao, J., Qiu, J., Liang, G. & Dong, Z. Y. Cooperative wind farm control with deep reinforcement learning and knowledge-assisted learning. *IEEE Trans. Indust. Inform.***16** , 6912–6921 (2020). 17. Dalili, N., Edrisy, A. & Carriveau, R. A review of surface engineering issues critical to wind turbine performance. *Renew. Sustain. Energy Rev.***13** , 428–438 (2009). 18. Chamorro, L. P., Arndt, R. & Sotiropoulos, F. Drag reduction of large wind turbine blades through riblets: evaluation of riblet geometry and application strategies. *Renew. Energy***50** , 1095–1105 (2013). 19. Gao, H., Kaltenbach, S. & Koumoutsakos, P. Generative learning for forecasting the dynamics of high-dimensional complex systems. *Nat. Commun.***15** , 8904 (2024). 20. Bae, H. J. & Koumoutsakos, P. Scientific multi-agent reinforcement learning for wall-models of turbulent flows. *Nat. Commun.***13** , 1443 (2022). 21. Zolman, N., Lagemann, C., Fasel, U., Kutz, J. N. & Brunton, S. L. Sindy-rl for interpretable and efficient model-based reinforcement learning. *Nat. Commun.***16** , 10714 (2025). 22. Vinyals, O. et al. Grandmaster level in Starcraft II using multi-agent reinforcement learning. *Nature***575** , 350–354 (2019). 23. Fan, D., Yang, L., Wang, Z., Triantafyllou, M. S. & Karniadakis, G. E. Reinforcement learning for bluff body active flow control in experiments and simulations. *Proc Natl Acad. Sci. USA***117** , 26091–26098 (2020). 24. Bhola, S., Pawar, S., Balaprakash, P. & Maulik, R. Multi-fidelity reinforcement learning framework for shape optimization. *J. Comp. Phys.***482** , 112018 (2023). 25. Schulman, J., Wolski, F., Dhariwal, P., Radford, A. & Klimov, O. Proximal policy optimization algorithms. Preprint at [https://arxiv.org/abs/1707.06347](http://arxiv.org/abs/1707.06347) (2017). 26. Lillicrap, T. P. et al. Continuous control with deep reinforcement learning. In *4th International Conference on Learning Representations* (ICLR, 2016). 27. Fujimoto, S., Hoof, H. & Meger, D. Addressing function approximation error in actor-critic methods. *Proc. 35th International Conference on Machine Learning* 1587–1596 (ICML, 2018). 28. Son, S., Zheng, L., Sullivan, R., Qiao, Y.-L. & Lin, M. Gradient informed proximal policy optimization. In *NIPS '23: Proc. 37th International Conference on Neural Information Processing Systems* Vol. 36, 8788–8814 (NeurIPS, 2023). 29. Xu, J. et al. Accelerated policy learning with parallel differentiable simulation. In *10th International Conference on Learning Representations* (ICLR, 2022). 30. Blanchard, A. & Sapsis, T. P. Stabilization of unsteady flows by reduced-order control with optimally time-dependent modes. *Phys. Rev. Fluids***4** , 053902 (2019). 31. Suárez, P. et al. Active flow control for drag reduction through multi-agent reinforcement learning on a turbulent cylinder at Re *D* = 3900.*Flow Turbul. Combust* .[https://doi.org/10.1007/s10494-025-00642-x](https://doi.org/10.1007/s10494-025-00642-x) (2025). 32. Maulik, R., Fukami, K., Ramachandra, N., Fukagata, K. & Taira, K. Probabilistic neural networks for fluid flow surrogate modeling and data recovery. *Phys. Rev. Fluids***5** , 104401 (2020). 33. Duraisamy, K., Iaccarino, G. & Xiao, H. Turbulence modeling in the age of data. *Ann. Rev. Fluid Mech.***51** , 357–377 (2019). 34. Liu, Q. et al. Unsteady control of supersonic turbulent cavity flow based on resolvent analysis. *J. Fluid Mech.***925** , A5 (2021). 35. Semeraro, O., Bagheri, S., Brandt, L. & Henningson, D. S. Feedback control of three-dimensional optimal disturbances using reduced-order models. *J. Fluid Mech.***677** , 63–102 (2011). 36. List, B., Chen, L.-W., Bali, K. & Thuerey, N. Differentiability in unrolled training of neural physics simulators on transient dynamics. *Comp. Meth. Appl. Mech. Eng.***433** , 117441 (2025). 37. Koehler, F. Niedermayr, S. & Thuerey, N. Apebench: a benchmark for autoregressive neural emulators of PDEs. In *38th Conference on Neural Information Processing Systems* 120310–120252 (NeurIPS, 2024). 38. Cremades, A. et al. Identifying regions of importance in wall-bounded turbulence through explainable deep learning. *Nat. Commun.***15** , 3864 (2024). 39. Lozano-Durán, A. & Bae, H. J. Machine learning building-block-flow wall model for large-eddy simulation. *J. Fluid Mech.***963** , A35 (2023). 40. Lagemann, K., Lagemann, C. & Mukherjee, S. Invariance-based learning of latent dynamics. In *International Conference on Learning Representations* 38843–38887 (ICLR, 2024). 41. Martínez-Sánchez, Á, Arranz, G. & Lozano-Durán, A. Decomposing causality into its synergistic, unique, and redundant components. *Nat. Commun.***15** , 9296 (2024). 42. Lagemann, K., Lagemann, C., Taschler, B. & Mukherjee, S. Deep learning of causal structures in high dimensions under data limitations. *Nat. Mach. Intell.***5** , 1306–1316 (2023). 43. Cranmer, M. et al. Discovering symbolic models from deep learning with inductive biases. In *34th Conference on Neural Information Processing Systems* Vol. 33, 17429–17442 (NeurIPS, 2020). 44. Towers, M. et al. Gymnasium: A standard interface for reinforcement learning environments. Preprint [https://arxiv.org/abs/2407.17032](http://arxiv.org/abs/2407.17032) (2024). 45. Institute of Aerodynamics. m-AIA. *Zenodo*[https://doi.org/10.5281/zenodo.13350586](https://doi.org/10.5281/zenodo.13350586) (2024). 46. Geier, M., Schönherr, M., Pasquali, A. & Krafczyk, M. The cumulant lattice Boltzmann equation in three dimensions: theory and validation. *Comput. Math Appl.***70** , 507–547 (2015). 47. Eitel-Amor, G., Meinke, M. & Schröder, W. A lattice-Boltzmann method with hierarchically refined meshes. *Comput. Fluids***75** , 127–139 (2013). 48. Albers, M., Shao, X. & Schröder, W. Energy efficient actuated drag reduced compressible turbulent flat plate flow. *Intl J. Heat Fluid Flow***106** , 109314 (2024). 49. Lagemann, E., Albers, M., Lagemann, C. & Schröder, W. Impact of Reynolds number on the drag reduction mechanism of spanwise travelling surface waves. *Flow Turbul. Combust.***113** , 27–40 (2024). 50. Mäteling, E. & Schröder, W. Analysis of spatiotemporal inner-outer large-scale interactions in turbulent channel flow by multivariate empirical mode decomposition. *Phys. Rev. Fluids***7** , 034603 (2022). 51. Mäteling, E., Klaas, M. & Schröder, W. Detection of small-scale/large-scale interactions in turbulent wall-bounded flows. *Phys. Rev. Fluids***5** , 114610 (2020). 52. Fischer, P. F., Lottes, J. W. & Kerkemeier, S. G. *Nek5000*[http://nek5000.mcs.anl.gov](http://nek5000.mcs.anl.gov) (Nek5000, 2008). 53. Maday, Y. & Patera, A. T. Spectral element methods for the incompressible Navier–Stokes equations. In *State-of-the-Art Surveys on Computational Mechanics* (eds Noor, A. K. & Oden, J. T.) 71–143 (American Society of Mechanical Engineers, 1989). 54. Bradbury, J. et al. *JAX: Composable Transformations of Python+NumPy Programs* (GitHub, 2018). 55. Lange, R. T. *gymnax: A JAX-based Reinforcement Learning Environment Library* (GitHub, 2022). 56. Lu, C. *PureJaxRL: End-to-End RL Training in Pure Jax* (GitHub, 2023). 57. Bezgin, D. A., Buhendwa, A. B. & Adams, N. A. Jax-Fluids: a fully-differentiable high-order computational fluid dynamics solver for compressible two-phase flows. *Comput. Phys. Commun.***282** , 108527 (2023). 58. Bezgin, D. A., Buhendwa, A. B. & Adams, N. A. Jax-Fluids 2.0: towards HPC for differentiable CFD of compressible two-phase flows. *Comput. Phys. Commun.***308** , 109433 (2025). 59. Ham, D. A. et al. *Firedrake User Manual* 1st edn (Imperial College London, 2023). 60. Raffin, A. et al. Stable-baselines3: reliable reinforcement learning implementations. *J. Mach. Learn. Res.***22** , 1–8 (2021). 61. Bou, A. et al. TorchRL: a data-driven decision-making library for PyTorch. In *12th International Conference on Learning Representations* (ICLR, 2024). 62. Huang, S. et al. CleanRL: high-quality single-file implementations of deep reinforcement learning algorithms. *J. Mach. Learn. Res.***23** , 1–18 (2022). 63. Wang, Y., Atzori, M. & Vinuesa, R. Opposition control applied to turbulent wing sections. *J. Fluid Mech.***1010** , A29 (2025). 64. Lagemann, C. et al. HydroGym: a multi-solver reinforcement learning platform for fluid dynamics. *Zenodo*[https://doi.org/10.5281/zenodo.21062222](https://doi.org/10.5281/zenodo.21062222) (2026). ## Acknowledgements We thank A. Blom-Schieber for valuable discussions. We also thank the anonymous referees for their constructive comments, which improved the manuscript. We would like to thank developers of—and contributors to—the open-source software on which HydroGym was built, and with which the presented results were obtained, including Firedrake, PyTorch, JAX, Stable-Baselines3 and TorchRL. ## Funding This work was supported by the National Science Foundation AI Institute in Dynamic Systems grant no. 2112085 and The Boeing Company (C.L., S.M., S.A., E.L., N.Z., S.L.B.). S.L.B. acknowledges support from the Army Research Office (grant no. W911NF-19-1-0045). C.L. acknowledges support from the German Research Foundation within the Walter Benjamin fellowship (LA 5508/1-1). E.L. acknowledges support from the German Research Foundation within the Walter Benjamin fellowship (MA 10764/1-1). M.R. acknowledges support from the German Research Foundation within the Walter Benjamin fellowship (RU 2771/1-1). L.P. acknowledges support from the German Research Foundation (project 326472365). M.M. and W.S. acknowledge support from the German Research Foundation (project no. SCHR309/68-1,2). N.A.A. acknowledges funding through ERC Advanced Grant (project no. 101094463; GENUFASD). R.V. acknowledges financial support from the University of Michigan and the European Research Council (grant agreement no. 2021-CoG-101043998; DEEPCONTROL). The views and opinions expressed are those of the authors only, and do not necessarily reflect those of the European Union or the European Research Council; neither the European Union nor the granting authority can be held responsible for them. We acknowledge the Gauss Centre for Supercomputing e.V. for providing computing time on the GCS Supercomputers. ## Ethics declarations ### Competing interests The authors declare no competing interests. ## Peer review ### Peer review information *Nature* thanks Clara Marika Velte and the other, anonymous, reviewer(s) for their contribution to the peer review of this work. [Peer reviewer reports](/articles/s41586-026-10917-6#MOESM2) are available. ## Additional information **Publisher’s note** Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. ## Extended data figures and tables ### [Extended Data Fig. 1 The HydroGym Environment Suite.](/articles/s41586-026-10917-6/figures/5) The HydroGym platform provides standardized access to over 61 benchmarked fluid dynamics environments through a unified Gymnasium-compatible interface. The comprehensive environment gallery showcases diverse flow configurations spanning two-dimensional and three-dimensional scenarios including cylinder flows, cavity flows, airfoils, channel flows, and turbulent cases across multiple Reynolds number regimes. This standardized approach enables consistent benchmarking and comparison of reinforcement learning algorithms across a vast spectrum of active flow control problems. ### [Extended Data Fig. 2 Reinforcement learning performance across two-dimensional flow control environments.](/articles/s41586-026-10917-6/figures/6) Training curves (left) show convergence behaviour for PPO, DDPG, and TD3 algorithms across three benchmark scenarios including **a** fluidic pinball (*R**e* = 100), **b** open cavity (*R**e* = 4,200), and **c** cylinder (*R**e* = 3,900) environments. Test rollouts (centre) demonstrate control effectiveness, revealing the agents’ ability to manage canonical two-dimensional phenomena including vortex shedding and Kelvin-Helmholtz instabilities. Flow visualizations (right) highlight the sophisticated wake manipulation and flow control achieved through learned policies. Training reward curves show the mean over *n* = 3 independent training runs with different random seeds; shaded bands denote ± 1 s.d. across seeds. Test rollouts represent a single policy rollout sequence. ### [Extended Data Fig. 3 Transfer learning capabilities across flow configurations and conditions.](/articles/s41586-026-10917-6/figures/7) Four transfer scenarios demonstrate knowledge generalization: **a** Reynolds number scaling (*R**e* = 200 → 1,000, **b** *R**e* = 1,000 → 3,900), **c** geometric transfer (circular to square cylinder), and **d** dimensional scaling (2D to 3D cylinder). Training curves compare standard PPO training with transfer learning (finetuned and zeroshot). Finetuned policies consistently achieve faster convergence, requiring approximately half the training episodes of baseline methods. Test rollouts confirm that improved training efficiency translates to effective control performance, with drag reduction across all transfer scenarios. Training reward curves show the mean over *n* = 3 independent training runs with different random seeds; shaded bands denote ± 1 s.d. across seeds. Test rollouts represent a single policy rollout sequence. ## Supplementary information ### [Supplementary Information (download PDF )](https://media.springernature.com/original/springer-static/esm/art%3A10.1038%2Fs41586-026-10917-6/MediaObjects/41586_2026_10917_MOESM1_ESM.pdf) Supplementary Figs. 1–24, Tables 1–6 and Supplementary Sections 1–5 (1, Extended Introduction; 2, Extended literature review; 3, HydroGym—numerical backends and implementation; 4, benchmarked RL agents; 5, environments; References. ## Rights and permissions **Open Access** This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit [http://creativecommons.org/licenses/by-nc-nd/4.0/](http://creativecommons.org/licenses/by-nc-nd/4.0/). ## About this article ### Cite this article Lagemann, C., Mokbel, S., Gondrum, M. *et al.* The HydroGym reinforcement learning platform for fluid dynamics. *Nature* (2026). https://doi.org/10.1038/s41586-026-10917-6 - Received: - Accepted: - Published: - Version of record: - DOI: https://doi.org/10.1038/s41586-026-10917-6