K-Fold Cross-Validation and Data Leakage in the Context of Climatology Forecasts

A review by Kapoor and Narayanan found that 294 published research studies across 17 domains produced overly optimistic results because of data leakage, a problem the article examines in the context of k-fold cross-validation for climatology temperature-anomaly forecasts. The article details how leakage arises from poor separation of training and test sets, proxy variables tied to the outcome, spatial bias, non-independent samples, and preprocessing parameters such as dimensionality reduction or feature normalization estimated on the full dataset before splitting into folds. It recommends estimating preprocessing parameters from the training set only and applying them to both training and test data, and examines the Facebook Prophet model on the HadCRUT5 dataset for forecasting global temperature anomalies.

The previous articles 34 and 31 provided a detailed overview of forecasting algorithms in the context of global temperature anomalies, as well as imputation techniques for climatology data. This article explains aspects of k-fold cross-validation CV and the problem of data leakage, and examines the Facebook Prophet model for forecasting temperature anomalies using the HadCRUT5 dataset. As noted by Gorriz et al. 1 , machine learning is a technique that can capture complex patterns and has frequently been applied to predictive inference on data from multiple sources. K-fold CV is an approach for assessing the probability that a machine learning model’s result occurs by chance, and may perform slightly better than standard hypothesis testing. Statistical inference using machine learning is generally carried out through predictive inference: datasets are split into CV folds, and the ability of the model to generalize on unseen data is assessed. However, a frequently occurring problem when applying machine learning techniques is data leakage, which can lead to errors as part of data analysis. Kapoor and Narayanan 2 review published research studies and report that 294 of those across 17 domains yielded overly optimistic results, attributed to data leakage. This requires careful consideration when interpreting the analyses and ensuring reproducibility within a specific domain. The authors also note that correcting for data leakage may be crucial. Reference 2 notes various interrelated aspects, such as lack of clean separation between training and test datasets or unintentional use of a proxy variable related to the outcome that may lead to data target leakage. Spatial bias, which refers to choosing test data from one geographic location but reporting the model’s performance in another geographic location, is mentioned as an example by the authors. Similarly, Sasse et al. 3 note that machine learning offers effective tools, and its wide applicability comes from making predictions for individual samples, across domains, including climatology. Without proper implementation and evaluation, machine learning pipelines can be affected by data leakage, and are unable to generalize to new data. This can have unintended societal implications. Data leakage, as noted above, is one of the most frequently encountered errors when applying machine learning techniques and occurs when the model gets to learn information about the test data it should not have seen. This indicates that any estimate of the error needs to be carefully examined. Since real test data is often unavailable, machine learning practitioners hold out a portion of the data as a test set to evaluate how well the model generalizes as part of CV. In k-fold CV Figure 1 , the model makes efficient use of the available data, by splitting it into k equally sized, non-overlapping subsets called folds . In each iteration, one of the subsets is used as the test dataset, and the other subsets are used for model training. The mean performance over all folds gives an estimate of how well the model generalizes on unseen data. Data leakage can also arise when the samples are not independent of each other, posing a major challenge during data preparation. When samples can be assumed to be independent and identically distributed IID , a random split into training and test sets may suffice, as also noted by Bouke et al. 32 . In addition, data leakage can arise due to model preprocessing parameters, especially when applying techniques such as dimensionality reduction or feature normalization to the entire dataset before splitting it into CV folds. This scenario can be prevented by estimating the preprocessing parameters from the training set only and then applying them to the training and test data Sasse et al. 3 . The subsequent section explains the applicability of k-fold CV towards forecasting global temperature anomalies. Reference 35 notes that Prophet is a forecasting technique with an additive model structure that separates trend from seasonal patterns or holiday effects, allowing for refining the model based on domain knowledge. The Prophet model is formulated as follows Chhabra et al. 16 where Y t is the observed time series value at time t, g t is the trend term that captures non-periodic changes in the time series, s t is the seasonal component representing weekly or yearly patterns, h t accounts for the effects of holidays and ε t is the residual error. The trend component g t captures the long-term progression of the time series, separating it from the seasonal fluctuations. The changepoints capture sudden changes in the data. Linear trend model 35 Where k represents the base growth rate, δ holds the rate changes at each changepoint, m represents the intercept, γ ensures continuity, a t represents the indicator vector corresponding to each changepoint. For the time series that approaches a maximum value, Prophet models it using a logistic growth function given by Where C represents the carrying capacity, the upper limit the series can approach, and this model tracks the S-shaped growth pattern. Seasonal fluctuations are patterns that occur at regular intervals. Prophet models them with a Fourier series, which is a periodic function as a sum of sine and cosine terms. The seasonal component is defined as Where N represents the number of Fourier terms, P represents the length of the seasonal cycle, aₙ and bₙ are Fourier coefficients estimated from the data. In recent years, climate change has led to notable shifts in sea levels both globally and locally. Several factors related to ocean and climate, such as temperature and ocean heat, directly affect sea level rise. In this regard, several published research studies describe how the Prophet model can be applied in the context of climate research and forecasting El Motaki et al. 5 . Focusing on sea levels, Elneel et al. 6 investigate time series models, namely autoregressive integrated moving average ARIMA and Facebook Prophet, whereas an earlier study by the same authors Elneel et al. 7 investigates the applicability of ARIMA for forecasting global mean sea level. In another study, Elneel et al. 8 explain that sea level rise is among the major challenges posed by climate change and has attracted considerable attention in recent decades. The authors summarize measures for mitigating sea level rise while observing that factors resulting from global climate change, such as rising temperatures, have led to direct and indirect effects on sea levels across various spatial scales. Turning to temperature, Al-Hasani and Al-Qaraguli 9 emphasize that accurate forecasts of seasonal temperature are important for adapting to climate change and planning crop yields in dry regions, especially deserts. The authors propose a data-driven method to forecast seasonal temperatures in Niamey with the Prophet model. In the context of rainfall prediction, El Motaki et al. 5 point out that accurate forecasts are important for water resource management and agricultural planning. They suggest that traditional forecasting techniques have difficulty modeling non-linear behavior and complex seasonal patterns, and propose a hybrid approach based on the Prophet model with a fuzzy clustering algorithm that integrates holiday events into the modeling process. Similarly, Fronzi et al. 10 observe that predicting water availability has become increasingly important in recent decades, mainly because of growing human demand and climate change, which affect groundwater resources, causing them to decline. To address this, they propose a framework to model hydrological parameters that uses Prophet in a Python environment. Their results are assessed using measures such as mean absolute percentage error MAPE , mean absolute error MAE and root mean square error RMSE , along with the correlation. On a similar theme, Basak et al. 11 explain that reliable drought prediction may be necessary for early warning of drought hazards and for managing water and the environment. The authors evaluate how well the Prophet model performs in predicting meteorological drought using the standardized precipitation index, contrasting the model performance with support vector machine SVM and multiple linear regression MLR models. In the context of climate forecasting, accurately predicting daily air temperature is important for various applications, as noted in the previous articles 31 and 34 . Along these lines, Naz et al. 12 explore the effectiveness of the NeuralProphet model, an extension of the Prophet model with an added neural network, to forecast minimum temperature more accurately for Chengdu. In another domain, Barnor et al. 13 describe how weather derivatives can help protect against weather-related fluctuations in sales volume and revenue. To price these contracts, one must predict the underlying weather variable over the relevant period of risk, and in this regard, the authors report forecast performance for a group of locations using a box plot. In another example outside of climatology, Dash et al. 14 discuss NeuralProphet, a modular and interpretable hybrid approach that improves pandemic forecasts by adding two neural network components: an auto-regressor and a lagged regressor. The authors also explain the use of a deep auto-regressor neural network in the implementation of these models. In contrast to Prophet, this approach may be applicable to multivariate multistep prediction. Coming back to the Prophet model itself, as noted at the beginning of this section, Sardar et al. 15 discuss this model as an open-source framework, built on an additive model, for forecasting time series data. Prophet fits non-linear trends together with daily, weekly or yearly seasonal patterns, accounting for holiday effects. In addition to producing forecasts, the Prophet model can also handle missing values and identify anomalies represented in the flow diagram above . In the context of forecasting COVID-19, Sardar et al. 15 also discuss ARIMA and Prophet models, as well as extreme gradient boosting, while Dash et al. 17 use the Prophet model for a similar application. In another example, Meng and Dou 18 point out that accurate tracking of carbon trading prices can provide reference indicators to governments and help reduce risks linked to carbon emissions. Based on the Prophet model, the long short-term memory LSTM neural network and the backpropagation neural network, they propose an ensemble learning model that forecasts carbon trading prices from market data. Studies in the related field of power systems also show the applicability of these models, for example, a hybrid LSTM-NeuralProphet model Shohan et al. 19 and the Prophet model Yildiz and Korkut 20 . Using vitamin D data from Google Trends as source information for the Prophet model, Sivaramakrishnan et al. 21 also describe the trend and seasonality components extracted using the model. Coming back to climate research, Alif et al. 22 apply Prophet to forecast rainfall considering climate variability and extreme rainfall events while quantifying the RMSE and coefficient of determination. Likewise, Abdel Rahman and Badawy 23 use Prophet to predict the number of sunspots for solar cycle 25, observing that solar magnetic activity is periodic, repeating approximately every eleven years. In this regard, the equation for RMSE is given as For air temperature, Toharudin et al. 24 note its importance in the context of climatology, and propose LSTM and Prophet for predicting it in Bandung. Thiyagarajan et al. 25 apply the Prophet model to time series forecasting and anomaly detection in a sewer air temperature sensor system, suggesting reasonable performance. Along similar lines, Haris et al. 26 propose LSTM and Prophet in the context of air temperature forecasting and climate change in Jakarta, Indonesia. Elseidi 27 proposes a combined Prophet-ARIMA framework for predicting temperature data at hourly or daily intervals that may have complex seasonal patterns and short-term dynamics influencing environmental systems. In hydrology, Islam et al. 28 assess how well the Prophet model performs in the context of river stage forecasting, comparing its performance across multiple lead times and hydrometeorological inputs. Hossain et al. 29 emphasize that early prediction of rainfall is important for the planning of agriculture, water infrastructure and socio-economic development, and investigate the applicability of Prophet and six other machine learning regression algorithms for monthly rainfall prediction on a ten-year horizon in the Brisbane River catchment. Olaleye et al. 30 present a decomposition-based hybrid Prophet-LSTM framework for drought forecasting. Finally, a closely related aspect concerns quantifying uncertainty. Hossain et al. 33 note that global mean sea level rise is key to assessing the impacts of climate change and, in this regard, analyze 32 years of high-resolution satellite altimetry records to study historical trends and forecast sea levels using the Prophet model, suggesting this to be a conservative baseline. HadCRUT5 is a gridded global historical surface temperature dataset that records near-surface temperature changes since January 1850 Morice et al. 4 . Figure 3 above shows a sharper rise approximately after 1980 compared with the earlier years. In this article, the HadCRUT5 temperature anomaly data are analyzed using Prophet and plotted from 2010. The forecast predictions together with the quantified uncertainty are shown below in Figure 5. Multiplicative seasonality was selected since it was able to track the peaks better. The analysis in Figure 6 considers several scenarios pertaining to changepoint prior scale and seasonality prior scale hyperparameters, to quantify the root mean squared error. Data leakage is an important consideration that can affect the accuracy of the analysis: the information from the test set leaks into the model training process. K-fold CV is one of the ways of assessing how well the model does on unseen data. Prophet is a time series forecasting model, applied in the context of climatology and predicting global temperature anomalies. The model captures trend, seasonality effects, and holidays as a summation, each of these components can be individually analyzed. Several scenarios in regard to minimizing the root mean squared error RMSE were analyzed through a Prophet grid search. All references have been cited appropriately. Claude was used to check the grammar, language and textual coherence of the article, and to assist with the experimental setup. 1 J. M. Gorriz, R. Martin-Clemente, F. Segovia, J. Ramirez, A. Ortiz and J. Suckling 2026 . Is K-fold cross validation the best model selection method for machine learning? Information Fusion, 135, Article 104404. https://doi.org/10.1016/j.inffus.2026.104404 https://doi.org/10.1016/j.inffus.2026.104404 2 S. Kapoor and A. Narayanan 2023 . Leakage and the reproducibility crisis in machine-learning-based science. Patterns, 4 9 , Article 100804. https://doi.org/10.1016/j.patter.2023.100804 https://doi.org/10.1016/j.patter.2023.100804 3 L. Sasse, E. Nicolaisen-Sobesky, J. Dukart, S. B. Eickhoff, M. Götz, S. Hamdan, V. Komeyer, A. Kulkarni, J. M. Lahnakoski, B. C. Love, F. Raimondo and K. R. Patil 2025 . Overview of leakage scenarios in supervised machine learning. Journal of Big Data, 12, Article 135. https://doi.org/10.1186/s40537-025-01193-8 https://doi.org/10.1186/s40537-025-01193-8 4 C. P. Morice, J. J. Kennedy, N. A. Rayner, J. P. Winn, E. Hogan, R. E. Killick et al. 2021 . An updated assessment of near-surface temperature change from 1850: The HadCRUT5 data set. Journal of Geophysical Research: Atmospheres, 126, Article e2019JD032361. https://doi.org/10.1029/2019JD032361 https://doi.org/10.1029/2019JD032361 5 S. El Motaki, A. El-Fengour and H. El Motaki 2026 . A combined approach to precipitation forecasting: Enhancing FB-Prophet with fuzzy clustering to capture sudden changes and seasonal patterns in climate data. Journal of Forecasting, 45 1 , 179–193. https://doi.org/10.1002/for.70036 https://doi.org/10.1002/for.70036 6 L. Elneel, M. S. Zitouni, H. Mukhtar and H. Al-Ahmad 2024 . Examining sea levels forecasting using autoregressive and prophet models. Scientific Reports, 14 1 , Article 14337. https://doi.org/10.1038/s41598-024-65184-0 https://doi.org/10.1038/s41598-024-65184-0 7 L. Elneel, M. S. Zitouni, H. Mukhtar and H. Al-Ahmad 2023 . Forecasting global mean sea level rise using autoregressive models. In 2023 30th IEEE International Conference on Electronics, Circuits and Systems ICECS pp. 1–4 . https://doi.org/10.1109/ICECS58634.2023.10382721 https://doi.org/10.1109/ICECS58634.2023.10382721 8 L. Elneel, M. S. Zitouni, H. Mukhtar, P. Galli and H. Al-Ahmad 2024 . Exploring key aspects of sea level rise and their implications: An overview. Water, 16 3 , Article 388. https://doi.org/10.3390/w16030388 https://doi.org/10.3390/w16030388 9 A. T. Al-Hasani and M. Al-Qaraguli 2025 . Seasonal temperature prediction in Niamey: A Prophet model approach. EDRAAK, 2025, 7–11. https://doi.org/10.70470/EDRAAK/2025/002 https://doi.org/10.70470/EDRAAK/2025/002 10 D. Fronzi, G. Narang, A. Galdelli, A. Pepi, A. Mancini and A. Tazioli 2024 . Towards groundwater-level prediction using Prophet forecasting method by exploiting a high-resolution hydrogeological monitoring system. Water, 16 1 , Article 152. https://doi.org/10.3390/w16010152 https://doi.org/10.3390/w16010152 11 A. Basak, A. T. M. S. Rahman, J. Das, T. Hosono and O. Kisi 2022 . Drought forecasting using the Prophet model in a semi-arid climate region of western India. Hydrological Sciences Journal, 67 9 , 1397–1417. https://doi.org/10.1080/02626667.2022.2082876 https://doi.org/10.1080/02626667.2022.2082876 12 F. Naz, A. A. Khan and J. Shao 2024 . Neural Prophet model for enhanced air temperature forecasting in Chengdu. In 2024 8th Asian Conference on Artificial Intelligence Technology ACAIT pp. 1235–1239 . https://doi.org/10.1109/ACAIT63902.2024.11022307 https://doi.org/10.1109/ACAIT63902.2024.11022307 13 N. Barnor, M. Kampouridis and P. Kanellopoulos 2026 . Machine learning for temperature forecasting in weather derivatives. Artificial Intelligence Review, 59 6 , Article 144. https://doi.org/10.1007/s10462-026-11539-0 https://doi.org/10.1007/s10462-026-11539-0 14 S. Dash, S. K. Giri, S. Mallik, S. K. Pani, M. A. Shah and H. Qin 2024 . Predictive healthcare modeling for early pandemic assessment leveraging deep auto regressor neural prophet. Scientific Reports, 14, Article 5287. https://doi.org/10.1038/s41598-024-55973-y https://doi.org/10.1038/s41598-024-55973-y 15 I. Sardar, M. A. Akbar, V. Leiva, A. Alsanad and P. Mishra 2023 . Machine learning and automatic ARIMA/Prophet models-based forecasting of COVID-19: Methodology, evaluation, and case study in SAARC countries. Stochastic Environmental Research and Risk Assessment, 37 1 , 345–359. https://doi.org/10.1007/s00477-022-02307-x https://doi.org/10.1007/s00477-022-02307-x 16 A. Chhabra, V. Sharma, A. Sharma and S. K. Singh 2025 . Time series-based framework for global temperature forecasting using Fb-Prophet. In N. Nedjah, G. Martínez Pérez, & B. B. Gupta Eds. , International Conference on Smart Systems and Advanced Computing SysCom 2022 pp. 147–157 . Springer. https://doi.org/10.1007/978-3-031-40905-9 16 https://doi.org/10.1007/978-3-031-40905-9 16 17 S. Dash, C. Chakraborty, S. K. Giri and S. K. Pani 2021 . Intelligent computing on time-series data analysis and prediction of COVID-19 pandemics. Pattern Recognition Letters, 151, 69–75. https://doi.org/10.1016/j.patrec.2021.07.027 https://doi.org/10.1016/j.patrec.2021.07.027 18 F. Meng and R. Dou 2024 . Prophet-LSTM-BP ensemble carbon trading price prediction model. Computational Economics, 63, 1805–1825. https://doi.org/10.1007/s10614-023-10384-5 https://doi.org/10.1007/s10614-023-10384-5 19 M. J. A. Shohan, M. O. Faruque and S. Y. Foo 2022 . Forecasting of electric load using a hybrid LSTM-Neural Prophet model. Energies, 15 6 , Article 2158. https://doi.org/10.3390/en15062158 https://doi.org/10.3390/en15062158 20 U. Yildiz and S. O. Korkut 2024 . Electricity consumption forecasting using the Prophet model in industry: A case study. In F. P. García Márquez, A. Jamil, I. S. Ramirez, S. Eken, & A. A. Hameed Eds. , Computing, Internet of Things and Data Analytics ICCIDA 2023 Studies in Computational Intelligence, Vol. 1145 . Springer. https://doi.org/10.1007/978-3-031-53717-2 10 https://doi.org/10.1007/978-3-031-53717-2 10 21 S. Sivaramakrishnan, C. R. Rathish, S. Premalatha and C. Niranjana 2023 . Introduction to AI technique and analysis of time series data using Facebook Prophet model. In A. Ahirwar, P. K. Shukla, M. Shrivastava, P. Maheshwary, & B. Gour Eds. , Innovative engineering with AI applications. Wiley. https://doi.org/10.1002/9781119792161.ch9 https://doi.org/10.1002/9781119792161.ch9 22 H. A. Alif, M. J. Mashrafi, U. Haldar et al. 2026 . Machine learning-driven rainfall forecasting model for sustainable and adaptive infrastructure planning. Discover Sustainability, 7, Article 376. https://doi.org/10.1007/s43621-026-02699-8 https://doi.org/10.1007/s43621-026-02699-8 23 H. I. Abdel Rahman and W. A. Badawy 2026 . Forecasting the number of sunspots for solar cycle 25 utilizing the Facebook Prophet model. Scientific Reports, 16 1 , Article 2066. https://doi.org/10.1038/s41598-025-33819-5 https://doi.org/10.1038/s41598-025-33819-5 24 T. Toharudin, R. S. Pontoh, R. E. Caraka, S. Zahroh, Y. Lee and R. C. Chen 2023 . Employing long short-term memory and Facebook Prophet model in air temperature forecasting. Communications in Statistics — Simulation and Computation, 52 2 , 279–290. https://doi.org/10.1080/03610918.2020.1854302 https://doi.org/10.1080/03610918.2020.1854302 25 K. Thiyagarajan, S. Kodagoda, N. Ulapane and M. Prasad 2020 . A temporal forecasting driven approach using Facebook’s Prophet method for anomaly detection in sewer air temperature sensor system. In 2020 15th IEEE Conference on Industrial Electronics and Applications ICIEA pp. 25–30 . https://doi.org/10.1109/ICIEA48937.2020.9248142 https://doi.org/10.1109/ICIEA48937.2020.9248142 26 M. D. Haris, D. Adytia and A. W. Ramadhan 2022 . Air temperature forecasting with long short-term memory and Prophet: A case study of Jakarta, Indonesia. In 2022 International Conference on Data Science and Its Applications ICoDSA pp. 251–256 . https://doi.org/10.1109/ICoDSA55874.2022.9862869 https://doi.org/10.1109/ICoDSA55874.2022.9862869 27 M. Elseidi 2024 . A hybrid Facebook Prophet-ARIMA framework for forecasting high-frequency temperature data. Modeling Earth Systems and Environment, 10, 1855–1867. https://doi.org/10.1007/s40808-023-01874-4 https://doi.org/10.1007/s40808-023-01874-4 28 M. T. Islam, M. A. A.-S. Ridoy, K. Yoshida, M. M. U. Faraji and A. K. M. Adham 2026 . Operational river stage forecasting in a data-scarce delta: Prophet model performance across multiple lead times and hydrometeorological input scenarios. Water Practice and Technology, 21 7 , 3097–3125. https://doi.org/10.2166/wpt.2026.365 https://doi.org/10.2166/wpt.2026.365 29 M. M. Hossain, A. H. M. F. Anwar, N. Garg, M. Prakash and M. Bari 2022 . Monthly rainfall prediction at catchment level with the Facebook Prophet model using observed and CMIP5 decadal data. Hydrology, 9 6 , Article 111. https://doi.org/10.3390/hydrology9060111 https://doi.org/10.3390/hydrology9060111 30 O. S. Olaleye, O. T. Faloye, O. E. Adeyeri, O. A. Akintola, A. T. Ogunrinde, B. A. Adabembe, T. E. Babalola and J. O. Akinremi 2026 . A decomposition-based hybrid Prophet–LSTM framework for SPEI-12 drought forecasting in Kano State, Nigeria. AgriEngineering, 8 8 , Article 307. https://doi.org/10.3390/agriengineering8080307 https://doi.org/10.3390/agriengineering8080307 31 S. Jayaraman 2026 . Imputation of missing climate data and vulnerability analysis: A survey. Medium. 32 M. A. Bouke, S. A. Zaid and A. Abdullah 2024 . Implications of data leakage in machine learning preprocessing: A multi-domain investigation Preprint . Research Square. https://doi.org/10.21203/rs.3.rs-4579465/v1 https://doi.org/10.21203/rs.3.rs-4579465/v1 33 S. Hossain, T. B. Hasan, H. Hasan, M. F. I. Sattique and M. F. Mridha 2026 . A machine learning approach for quantifying sea-level rise: A conservative baseline forecast through 2030 using Prophet architecture. In 2026 IEEE 2nd International Conference on Quantum Photonics, Artificial Intelligence & Networking QPAIN . IEEE. 34 S. Jayaraman 2026 . Global temperature forecasting and climate change: A survey. Medium. https://medium.com/p/2b1a6e8673da https://medium.com/p/2b1a6e8673da 35 M. Brenndoerfer 2025 . Prophet time series forecasting: Trend, seasonality. https://mbrenndoerfer.com/writing/prophet-time-series-forecasting-trend-seasonality-holiday-effects https://mbrenndoerfer.com/writing/prophet-time-series-forecasting-trend-seasonality-holiday-effects K-Fold Cross-Validation and Data Leakage in the Context of Climatology Forecasts https://pub.towardsai.net/k-fold-cross-validation-and-data-leakage-in-the-context-of-climatology-forecasts-cfbf9e884850 was originally published in Towards AI https://pub.towardsai.net on Medium, where people are continuing the conversation by highlighting and responding to this story.