cd /news/machine-learning/k-fold-cross-validation-and-data-lea… · home › topics › machine-learning › article
[ARTICLE · art-162560] src=pub.towardsai.net ↗ pub= topic=machine-learning verified=true sentiment=· neutral

K-Fold Cross-Validation and Data Leakage in the Context of Climatology Forecasts

A review by Kapoor and Narayanan found that 294 published research studies across 17 domains produced overly optimistic results because of data leakage, a problem the article examines in the context of k-fold cross-validation for climatology temperature-anomaly forecasts. The article details how leakage arises from poor separation of training and test sets, proxy variables tied to the outcome, spatial bias, non-independent samples, and preprocessing parameters such as dimensionality reduction or feature normalization estimated on the full dataset before splitting into folds. It recommends estimating preprocessing parameters from the training set only and applying them to both training and test data, and examines the Facebook Prophet model on the HadCRUT5 dataset for forecasting global temperature anomalies.

by read17 min views1 publishedNov 2, 2026

The previous articles [34] and [31] provided a detailed overview of forecasting algorithms in the context of global temperature anomalies, as well as imputation techniques for climatology data. This article explains aspects of k-fold cross-validation (CV) and the problem of data leakage, and examines the Facebook Prophet model for forecasting temperature anomalies using the HadCRUT5 dataset.

As noted by Gorriz et al. [1], machine learning is a technique that can capture complex patterns and has frequently been applied to predictive inference on data from multiple sources. K-fold CV is an approach for assessing the probability that a machine learning model’s result occurs by chance, and may perform slightly better than standard hypothesis testing. Statistical inference (using machine learning) is generally carried out through predictive inference: datasets are split into CV folds, and the ability of the model to generalize on unseen data is assessed.

However, a frequently occurring problem when applying machine learning techniques is data leakage, which can lead to errors as part of data analysis. Kapoor and Narayanan [2] review published research studies and report that 294 of those across 17 domains yielded overly optimistic results, attributed to data leakage. This requires careful consideration when interpreting the analyses and ensuring reproducibility within a specific domain. The authors also note that correcting for data leakage may be crucial.

Reference [2] notes various interrelated aspects, such as lack of clean separation between training and test datasets or unintentional use of a proxy variable related to the outcome that may lead to data (target) leakage. Spatial bias, which refers to choosing test data from one geographic location but reporting the model’s performance in another geographic location, is mentioned as an example by the authors.

Similarly, Sasse et al. [3] note that machine learning offers effective tools, and its wide applicability comes from making predictions for individual samples, across domains, including climatology. Without proper implementation and evaluation, machine learning pipelines can be affected by data leakage, and are unable to generalize to new data. This can have unintended societal implications. Data leakage, as noted above, is one of the most frequently encountered errors when applying machine learning techniques and occurs when the model gets to learn information about the test data it should not have seen. This indicates that any estimate of the error needs to be carefully examined.

Since real test data is often unavailable, machine learning practitioners hold out a portion of the data as a test set to evaluate how well the model generalizes as part of CV. In k-fold CV (Figure 1), the model makes efficient use of the available data, by splitting it into k equally sized, non-overlapping subsets (called folds). In each iteration, one of the subsets is used as the test dataset, and the other subsets are used for model training. The mean performance over all folds gives an estimate of how well the model generalizes on unseen data.

Data leakage can also arise when the samples are not independent of each other, posing a major challenge during data preparation. When samples can be assumed to be independent and identically distributed (IID), a random split into training and test sets may suffice, as also noted by Bouke et al. [32].

In addition, data leakage can arise due to model preprocessing parameters, especially when applying techniques such as dimensionality reduction or feature normalization to the entire dataset before splitting it into CV folds. This scenario can be prevented by estimating the preprocessing parameters from the training set only and then applying them to the training and test data (Sasse et al. [3]).

The subsequent section explains the applicability of k-fold CV towards forecasting global temperature anomalies.

Reference [35] notes that Prophet is a forecasting technique with an additive model structure that separates trend from seasonal patterns or holiday effects, allowing for refining the model based on domain knowledge.

The Prophet model is formulated as follows (Chhabra et al. [16]) where Y(t) is the observed time series value at time t, g(t) is the trend term that captures non-periodic changes in the time series, s(t) is the seasonal component representing weekly or yearly patterns, h(t) accounts for the effects of holidays and ε(t) is the residual error.

The trend component g(t) captures the long-term progression of the time series, separating it from the seasonal fluctuations. The changepoints capture sudden changes in the data.

Linear trend model [35] Where k represents the base growth rate, δ holds the rate changes at each changepoint, m represents the intercept, γ ensures continuity, a(t) represents the indicator vector corresponding to each changepoint.

For the time series that approaches a maximum value, Prophet models it using a logistic growth function given by Where C represents the carrying capacity, the upper limit the series can approach, and this model tracks the S-shaped growth pattern.

Seasonal fluctuations are patterns that occur at regular intervals. Prophet models them with a Fourier series, which is a periodic function as a sum of sine and cosine terms.

The seasonal component is defined as

Where N represents the number of Fourier terms, P represents the length of the seasonal cycle, aₙ and bₙ are Fourier coefficients estimated from the data.

In recent years, climate change has led to notable shifts in sea levels both globally and locally. Several factors related to ocean and climate, such as temperature and ocean heat, directly affect sea level rise. In this regard, several published research studies describe how the Prophet model can be applied in the context of climate research and forecasting (El Motaki et al. [5]).

Focusing on sea levels, Elneel et al. [6] investigate time series models, namely autoregressive integrated moving average (ARIMA) and Facebook Prophet, whereas an earlier study by the same authors (Elneel et al. [7]) investigates the applicability of ARIMA for forecasting global mean sea level. In another study, Elneel et al. [8] explain that sea level rise is among the major challenges posed by climate change and has attracted considerable attention in recent decades. The authors summarize measures for mitigating sea level rise while observing that factors resulting from global climate change, such as rising temperatures, have led to direct and indirect effects on sea levels across various spatial scales.

Turning to temperature, Al-Hasani and Al-Qaraguli [9] emphasize that accurate forecasts of seasonal temperature are important for adapting to climate change and planning crop yields in dry regions, especially deserts. The authors propose a data-driven method to forecast seasonal temperatures in Niamey with the Prophet model.

In the context of rainfall prediction, El Motaki et al. [5] point out that accurate forecasts are important for water resource management and agricultural planning. They suggest that traditional forecasting techniques have difficulty modeling non-linear behavior and complex seasonal patterns, and propose a hybrid approach based on the Prophet model (with a fuzzy clustering algorithm) that integrates holiday events into the modeling process.

Similarly, Fronzi et al. [10] observe that predicting water availability has become increasingly important in recent decades, mainly because of growing human demand and climate change, which affect groundwater resources, causing them to decline. To address this, they propose a framework to model hydrological parameters that uses Prophet in a Python environment. Their results are assessed using measures such as mean absolute percentage error (MAPE), mean absolute error (MAE) and root mean square error (RMSE), along with the correlation.

On a similar theme, Basak et al. [11] explain that reliable drought prediction may be necessary for early warning of drought hazards and for managing water and the environment. The authors evaluate how well the Prophet model performs in predicting meteorological drought using the standardized precipitation index, contrasting the model performance with support vector machine (SVM) and multiple linear regression (MLR) models.

In the context of climate forecasting, accurately predicting daily air temperature is important for various applications, as noted in the previous articles [31] and [34]. Along these lines, Naz et al. [12] explore the effectiveness of the NeuralProphet model, an extension of the Prophet model with an added neural network, to forecast minimum temperature more accurately for Chengdu.

In another domain, Barnor et al. [13] describe how weather derivatives can help protect against weather-related fluctuations in sales volume and revenue. To price these contracts, one must predict the underlying weather variable over the relevant period of risk, and in this regard, the authors report forecast performance for a group of locations using a box plot.

In another example outside of climatology, Dash et al. [14] discuss NeuralProphet, a modular and interpretable hybrid approach that improves pandemic forecasts by adding two neural network components: an auto-regressor and a lagged regressor. The authors also explain the use of a deep auto-regressor neural network in the implementation of these models. In contrast to Prophet, this approach may be applicable to multivariate multistep prediction.

Coming back to the Prophet model itself, as noted at the beginning of this section, Sardar et al. [15] discuss this model as an open-source framework, built on an additive model, for forecasting time series data. Prophet fits non-linear trends together with daily, weekly or yearly seasonal patterns, accounting for holiday effects. In addition to producing forecasts, the Prophet model can also handle missing values and identify anomalies (represented in the flow diagram above).

In the context of forecasting COVID-19, Sardar et al. [15] also discuss ARIMA and Prophet models, as well as extreme gradient boosting, while Dash et al. [17] use the Prophet model for a similar application.

In another example, Meng and Dou [18] point out that accurate tracking of carbon trading prices can provide reference indicators to governments and help reduce risks linked to carbon emissions. Based on the Prophet model, the long short-term memory (LSTM) neural network and the backpropagation neural network, they propose an ensemble learning model that forecasts carbon trading prices from market data.

Studies in the related field of power systems also show the applicability of these models, for example, a hybrid LSTM-NeuralProphet model (Shohan et al. [19]) and the Prophet model (Yildiz and Korkut [20]).

Using vitamin D data from Google Trends as source information for the Prophet model, Sivaramakrishnan et al. [21] also describe the trend and seasonality components extracted using the model. Coming back to climate research, Alif et al. [22] apply Prophet to forecast rainfall considering climate variability and extreme rainfall events while quantifying the RMSE and coefficient of determination. Likewise, Abdel Rahman and Badawy [23] use Prophet to predict the number of sunspots for solar cycle 25, observing that solar magnetic activity is periodic, repeating approximately every eleven years. In this regard, the equation for RMSE is given as

For air temperature, Toharudin et al. [24] note its importance in the context of climatology, and propose LSTM and Prophet for predicting it in Bandung. Thiyagarajan et al. [25] apply the Prophet model to time series forecasting and anomaly detection in a sewer air temperature sensor system, suggesting reasonable performance. Along similar lines, Haris et al. [26] propose LSTM and Prophet in the context of air temperature forecasting and climate change in Jakarta, Indonesia. Elseidi [27] proposes a combined Prophet-ARIMA framework for predicting temperature data (at hourly or daily intervals) that may have complex seasonal patterns and short-term dynamics influencing environmental systems. In hydrology, Islam et al. [28] assess how well the Prophet model performs in the context of river stage forecasting, comparing its performance across multiple lead times and hydrometeorological inputs.

Hossain et al. [29] emphasize that early prediction of rainfall is important for the planning of agriculture, water infrastructure and socio-economic development, and investigate the applicability of Prophet and six other machine learning regression algorithms for monthly rainfall prediction on a ten-year horizon in the Brisbane River catchment. Olaleye et al. [30] present a decomposition-based hybrid Prophet-LSTM framework for drought forecasting.

Finally, a closely related aspect concerns quantifying uncertainty. Hossain et al. [33] note that global mean sea level rise is key to assessing the impacts of climate change and, in this regard, analyze 32 years of high-resolution satellite altimetry records to study historical trends and forecast sea levels using the Prophet model, suggesting this to be a conservative baseline.

HadCRUT5 is a gridded global historical surface temperature dataset that records near-surface temperature changes since January 1850 (Morice et al. [4]). Figure 3 (above) shows a sharper rise approximately after 1980 compared with the earlier years. In this article, the HadCRUT5 temperature anomaly data are analyzed using Prophet and plotted from 2010. The forecast predictions together with the quantified uncertainty are shown below in Figure 5.

Multiplicative seasonality was selected since it was able to track the peaks better. The analysis in Figure 6 considers several scenarios pertaining to changepoint prior scale and seasonality prior scale hyperparameters, to quantify the root mean squared error.

Data leakage is an important consideration that can affect the accuracy of the analysis: the information from the test set leaks into the model training process. K-fold CV is one of the ways of assessing how well the model does on unseen data.

Prophet is a time series forecasting model, applied in the context of climatology and predicting global temperature anomalies. The model captures trend, seasonality effects, and holidays as a summation, each of these components can be individually analyzed. Several scenarios in regard to minimizing the root mean squared error (RMSE) were analyzed through a Prophet grid search.

All references have been cited appropriately. Claude was used to check the grammar, language and textual coherence of the article, and to assist with the experimental setup.

[1] J. M. Gorriz, R. Martin-Clemente, F. Segovia, J. Ramirez, A. Ortiz and J. Suckling (2026). Is K-fold cross validation the best model selection method for machine learning? Information Fusion, 135, Article 104404. https://doi.org/10.1016/j.inffus.2026.104404

[2] S. Kapoor and A. Narayanan (2023). Leakage and the reproducibility crisis in machine-learning-based science. Patterns, 4(9), Article 100804. https://doi.org/10.1016/j.patter.2023.100804

[3] L. Sasse, E. Nicolaisen-Sobesky, J. Dukart, S. B. Eickhoff, M. Götz, S. Hamdan, V. Komeyer, A. Kulkarni, J. M. Lahnakoski, B. C. Love, F. Raimondo and K. R. Patil (2025). Overview of leakage scenarios in supervised machine learning. Journal of Big Data, 12, Article 135. https://doi.org/10.1186/s40537-025-01193-8

[4] C. P. Morice, J. J. Kennedy, N. A. Rayner, J. P. Winn, E. Hogan, R. E. Killick et al. (2021). An updated assessment of near-surface temperature change from 1850: The HadCRUT5 data set. Journal of Geophysical Research: Atmospheres, 126, Article e2019JD032361. https://doi.org/10.1029/2019JD032361

[5] S. El Motaki, A. El-Fengour and H. El Motaki (2026). A combined approach to precipitation forecasting: Enhancing FB-Prophet with fuzzy clustering to capture sudden changes and seasonal patterns in climate data. Journal of Forecasting, 45(1), 179–193. https://doi.org/10.1002/for.70036

[6] L. Elneel, M. S. Zitouni, H. Mukhtar and H. Al-Ahmad (2024). Examining sea levels forecasting using autoregressive and prophet models. Scientific Reports, 14(1), Article 14337. https://doi.org/10.1038/s41598-024-65184-0

[7] L. Elneel, M. S. Zitouni, H. Mukhtar and H. Al-Ahmad (2023). Forecasting global mean sea level rise using autoregressive models. In 2023 30th IEEE International Conference on Electronics, Circuits and Systems (ICECS) (pp. 1–4). https://doi.org/10.1109/ICECS58634.2023.10382721

[8] L. Elneel, M. S. Zitouni, H. Mukhtar, P. Galli and H. Al-Ahmad (2024). Exploring key aspects of sea level rise and their implications: An overview. Water, 16(3), Article 388. https://doi.org/10.3390/w16030388

[9] A. T. Al-Hasani and M. Al-Qaraguli (2025). Seasonal temperature prediction in Niamey: A Prophet model approach. EDRAAK, 2025, 7–11. https://doi.org/10.70470/EDRAAK/2025/002

[10] D. Fronzi, G. Narang, A. Galdelli, A. Pepi, A. Mancini and A. Tazioli (2024). Towards groundwater-level prediction using Prophet forecasting method by exploiting a high-resolution hydrogeological monitoring system. Water, 16(1), Article 152. https://doi.org/10.3390/w16010152

[11] A. Basak, A. T. M. S. Rahman, J. Das, T. Hosono and O. Kisi (2022). Drought forecasting using the Prophet model in a semi-arid climate region of western India. Hydrological Sciences Journal, 67(9), 1397–1417. https://doi.org/10.1080/02626667.2022.2082876

[12] F. Naz, A. A. Khan and J. Shao (2024). Neural Prophet model for enhanced air temperature forecasting in Chengdu. In 2024 8th Asian Conference on Artificial Intelligence Technology (ACAIT) (pp. 1235–1239). https://doi.org/10.1109/ACAIT63902.2024.11022307

[13] N. Barnor, M. Kampouridis and P. Kanellopoulos (2026). Machine learning for temperature forecasting in weather derivatives. Artificial Intelligence Review, 59(6), Article 144. https://doi.org/10.1007/s10462-026-11539-0

[14] S. Dash, S. K. Giri, S. Mallik, S. K. Pani, M. A. Shah and H. Qin (2024). Predictive healthcare modeling for early pandemic assessment leveraging deep auto regressor neural prophet. Scientific Reports, 14, Article 5287. https://doi.org/10.1038/s41598-024-55973-y

[15] I. Sardar, M. A. Akbar, V. Leiva, A. Alsanad and P. Mishra (2023). Machine learning and automatic ARIMA/Prophet models-based forecasting of COVID-19: Methodology, evaluation, and case study in SAARC countries. Stochastic Environmental Research and Risk Assessment, 37(1), 345–359. https://doi.org/10.1007/s00477-022-02307-x

[16] A. Chhabra, V. Sharma, A. Sharma and S. K. Singh (2025). Time series-based framework for global temperature forecasting using Fb-Prophet. In N. Nedjah, G. Martínez Pérez, & B. B. Gupta (Eds.), International Conference on Smart Systems and Advanced Computing (SysCom 2022) (pp. 147–157). Springer. https://doi.org/10.1007/978-3-031-40905-9_16

[17] S. Dash, C. Chakraborty, S. K. Giri and S. K. Pani (2021). Intelligent computing on time-series data analysis and prediction of COVID-19 pandemics. Pattern Recognition Letters, 151, 69–75. https://doi.org/10.1016/j.patrec.2021.07.027

[18] F. Meng and R. Dou (2024). Prophet-LSTM-BP ensemble carbon trading price prediction model. Computational Economics, 63, 1805–1825. https://doi.org/10.1007/s10614-023-10384-5

[19] M. J. A. Shohan, M. O. Faruque and S. Y. Foo (2022). Forecasting of electric load using a hybrid LSTM-Neural Prophet model. Energies, 15(6), Article 2158. https://doi.org/10.3390/en15062158

[20] U. Yildiz and S. O. Korkut (2024). Electricity consumption forecasting using the Prophet model in industry: A case study. In F. P. García Márquez, A. Jamil, I. S. Ramirez, S. Eken, & A. A. Hameed (Eds.), Computing, Internet of Things and Data Analytics (ICCIDA 2023) (Studies in Computational Intelligence, Vol. 1145). Springer. https://doi.org/10.1007/978-3-031-53717-2_10

[21] S. Sivaramakrishnan, C. R. Rathish, S. Premalatha and C. Niranjana (2023). Introduction to AI technique and analysis of time series data using Facebook Prophet model. In A. Ahirwar, P. K. Shukla, M. Shrivastava, P. Maheshwary, & B. Gour (Eds.), Innovative engineering with AI applications. Wiley. https://doi.org/10.1002/9781119792161.ch9

[22] H. A. Alif, M. J. Mashrafi, U. Haldar et al. (2026). Machine learning-driven rainfall forecasting model for sustainable and adaptive infrastructure planning. Discover Sustainability, 7, Article 376. https://doi.org/10.1007/s43621-026-02699-8

[23] H. I. Abdel Rahman and W. A. Badawy (2026). Forecasting the number of sunspots for solar cycle 25 utilizing the Facebook Prophet model. Scientific Reports, 16(1), Article 2066. https://doi.org/10.1038/s41598-025-33819-5

[24] T. Toharudin, R. S. Pontoh, R. E. Caraka, S. Zahroh, Y. Lee and R. C. Chen (2023). Employing long short-term memory and Facebook Prophet model in air temperature forecasting. Communications in Statistics — Simulation and Computation, 52(2), 279–290. https://doi.org/10.1080/03610918.2020.1854302

[25] K. Thiyagarajan, S. Kodagoda, N. Ulapane and M. Prasad (2020). A temporal forecasting driven approach using Facebook’s Prophet method for anomaly detection in sewer air temperature sensor system. In 2020 15th IEEE Conference on Industrial Electronics and Applications (ICIEA) (pp. 25–30). https://doi.org/10.1109/ICIEA48937.2020.9248142

[26] M. D. Haris, D. Adytia and A. W. Ramadhan (2022). Air temperature forecasting with long short-term memory and Prophet: A case study of Jakarta, Indonesia. In 2022 International Conference on Data Science and Its Applications (ICoDSA) (pp. 251–256). https://doi.org/10.1109/ICoDSA55874.2022.9862869

[27] M. Elseidi (2024). A hybrid Facebook Prophet-ARIMA framework for forecasting high-frequency temperature data. Modeling Earth Systems and Environment, 10, 1855–1867. https://doi.org/10.1007/s40808-023-01874-4

[28] M. T. Islam, M. A. A.-S. Ridoy, K. Yoshida, M. M. U. Faraji and A. K. M. Adham (2026). Operational river stage forecasting in a data-scarce delta: Prophet model performance across multiple lead times and hydrometeorological input scenarios. Water Practice and Technology, 21(7), 3097–3125. https://doi.org/10.2166/wpt.2026.365

[29] M. M. Hossain, A. H. M. F. Anwar, N. Garg, M. Prakash and M. Bari (2022). Monthly rainfall prediction at catchment level with the Facebook Prophet model using observed and CMIP5 decadal data. Hydrology, 9(6), Article 111. https://doi.org/10.3390/hydrology9060111

[30] O. S. Olaleye, O. T. Faloye, O. E. Adeyeri, O. A. Akintola, A. T. Ogunrinde, B. A. Adabembe, T. E. Babalola and J. O. Akinremi (2026). A decomposition-based hybrid Prophet–LSTM framework for SPEI-12 drought forecasting in Kano State, Nigeria. AgriEngineering, 8(8), Article 307. https://doi.org/10.3390/agriengineering8080307

[31] S. Jayaraman (2026). Imputation of missing climate data and vulnerability analysis: A survey. Medium.

[32] M. A. Bouke, S. A. Zaid and A. Abdullah (2024). Implications of data leakage in machine learning preprocessing: A multi-domain investigation [Preprint]. Research Square. https://doi.org/10.21203/rs.3.rs-4579465/v1

[33] S. Hossain, T. B. Hasan, H. Hasan, M. F. I. Sattique and M. F. Mridha (2026). A machine learning approach for quantifying sea-level rise: A conservative baseline forecast through 2030 using Prophet architecture. In 2026 IEEE 2nd International Conference on Quantum Photonics, Artificial Intelligence & Networking (QPAIN). IEEE.

[34] S. Jayaraman (2026). Global temperature forecasting and climate change: A survey. Medium. https://medium.com/p/2b1a6e8673da [35] M. Brenndoerfer (2025). Prophet time series forecasting: Trend, seasonality. https://mbrenndoerfer.com/writing/prophet-time-series-forecasting-trend-seasonality-holiday-effects

K-Fold Cross-Validation and Data Leakage in the Context of Climatology Forecasts was originally published in Towards AI on Medium, where people are continuing the conversation by highlighting and responding to this story.

── more in #machine-learning 4 stories · sorted by recency
── more on @kapoor 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
→ Live at https://your-agent.zahid.host ✓
Get free account → Pricing
from €0/mo · no card required
LIVE [news/k-fold-cross-validat…] indexed:0 read:17min 2026-11-02 · —