Even after decades of meteorological advances, weather forecasting remains infamously difficult. Today’s most trusted models are rarely accurate beyond a two-week window due to a constant cascade of atmospheric and climatic variables. These unpredictable meteorological interactions are the foundations of chaos theory. The interdisciplinary field studies how even the world’s most predictable systems can still produce completely unpredictable results, and was first developed by MIT mathematician Edward Lorenz during the 1970s.
“He came up with this theoretical two-week limit where beyond that, we just can’t predict weather because the noise in the system grows so much that it overwhelms any signal you start with,” Judah Cohen, a climatologist also at MIT, explained.
According to Cohen, many climate researchers continue to view Lorenz’s two-week hurdle as insurmountable. And that assumption has largely proven true for nearly 50 years. Even today’s dynamical simulations and artificial intelligence models begin to break down when attempting to predict conditions two-to-six weeks in the future—a window known as subseasonal forecasting. Nicknamed the “predictability desert,” this range runs into slow-moving, highly variable forces like this year’s super El Niño event. Once those factors enter the equation, forecasting becomes more of a roll of the dice l than anything concrete. However, Cohen has had his suspicions about chaos theory’s limitations.
“I never really believed that completely,” he said. “In theory, the atmosphere can create an infinite number of patterns, but it likes to fall into these favorite patterns over and over again.”
To try and break this forecasting barrier, Cohen and his colleague Lester Mackey have spent years developing an AI system that not only minimizes forecasting errors, but learns from its inevitable mistakes through a method called postprocessing. With postprocessing, the AI improves by comparing the inaccuracies with existing historical data, and then adjusts from there.
Cohen’s team recently submitted three versions of their model featuring the postprocessing system the European Center for Medium-Range Weather Forecasts (ECMWF). Widely known as the world’s best forecasting hub, the ECMWF also hosts seasonal competitions that pit the latest prediction models against one another. Most of these systems have relied on historic climate averages to reach their conclusions, but this was the first time a group submitted a program with postprocessing capabilities.
Cohen and his colleagues reported their results in a study published today in the journal Nature. Their AI postprocessing integration blew the existing forecasting models out of the water. For the two-to-six-week subseasonal window, researchers doubled overall prediction accuracy. Such massive improvements could soon help the farmers and energy providers who already rely on subseasonal forecasts, but the benefits may even save lives.
“It also improves predictions of extreme weather events, whether it’s flooding or cold spells or something else,” said Cohen. “It’s really important to predict both, but extreme events might be the most impactful part of this for society.”
Many researchers have long presumed subseasonal forecasting to be a dead end, but the latest leap may clear the way to even bigger advances in the field.
“People have questioned if it’s even worth investing in models at all for this time range,” said Cohen. “[But] we consistently beat climatology averages with our models.”