SciML in the Wild: A Diagnostic Study of When Structural Priors Help and When They Hurt

A study evaluating Scientific Machine Learning models for macroeconomic forecasting found that less-constrained models like ARIMA and Neural ODEs consistently outperform more-constrained models such as Physics-Informed Neural Networks and Universal Differential Equations, indicating that structural priors can act as misregularizers when they do not match the data-generating process.

arXiv:2607.09684v1 Announce Type: new Abstract: Scientific Machine Learning SciML methods such as Neural Ordinary Differential Equations NODEs , Physics-Informed Neural Networks PINNs , and Universal Differential Equations UDEs are most effective when structural priors reflect reliable governing dynamics. We ask what happens when this assumption is violated. Using macroeconomic forecasting as a stress-test domain, we evaluate five model families, ARIMA, LSTM, NODE, PINN, and UDE, across 23 countries using sparse annual data, multiple temporal splits, and five random seeds. Our results show that none of the evaluated models achieve consistently strong forecasting performance, highlighting the difficulty of low-frequency macroeconomic prediction. However, a clear relative hierarchy emerges: less-constrained models, particularly ARIMA and NODE, consistently outperform more-constrained heuristic-prior models such as PINN and UDE. Rather than treating this as a rejection of SciML, we interpret it as a diagnostic result: structural priors can act as misregularizers when they do not match the data-generating process. We identify failure modes including prior misalignment, regime shifts, structural breaks, and optimization instability, and argue that SciML practitioners should test whether structure helps before assuming that more structure is beneficial.