# Fast Forecast Experiments Still Need to Wait for Outcomes > Source: > Published: 2026-10-19 18:02:28+00:00 A forecasting service can generate its next chart in seconds. Learning whether that chart was useful takes longer. If the target covers twelve future candles, most of its outcome does not exist when the prediction appears. A fast research loop needs to preserve that delay rather than quietly erase it during replay. Disclosure: An AI agent researched and drafted this article from my public project documentation and the linked primary sources. The timing examples and proposed checks are illustrative engineering recommendations, not new experimental results or investment recommendations. My [Traid / Traidence case study](https://asaday.co/projects/traid) describes a research platform with paper qualification ongoing. Each forecast generates twelve candidate paths over twelve future candles. Independent selectors cover eight markets and three timeframes. A forecast becomes training evidence only after its full twelve-candle outcome has matured. That last rule is as important to the experiment as the model that draws the paths. Traid builds around Kronos; my contribution is the terminal and the surrounding selection, persistence, replay, calibration, and evaluation system. The [Kronos paper, first submitted on August 2, 2025](https://arxiv.org/abs/2508.02739), describes a foundation model for financial candlestick sequences. Its [public repository](https://github.com/shiyu-coder/Kronos) provides model checkpoints and fine-tuning examples. This is an ongoing development in reusable forecasting models, rather than a new release this week. That availability changes where a small team can spend its effort. It can experiment with a selector, a research interface, or a data pipeline without first building a foundation model. But generating another candidate is different from collecting another independent observation of the world. I would use the speed to try more hypotheses in the development data, while keeping the rules for when evidence becomes available explicit. A research assistant can propose a feature or investigate a failed prediction. It should not turn an unresolved forecast into a completed lesson just because a later portion of the historical dataset is already on disk. Consider a simplified replay with one forecast issued at the close of every bar. A forecast issued at bar 100 is scored against bars 101 through 112. At bar 105, the replay may have five later bars available, but it still cannot know that forecast's full twelve-bar outcome. If the selector is retrained at bar 105 using the final score for the bar-100 forecast, future information has entered the experiment. Sorting rows by their forecast timestamp would not fix that. The relevant question is whether the label was available before the simulated training decision. For a proposed implementation, I would retain the forecast issue time, target interval, outcome-completion time, and the time the resolved data actually became available. These can differ when feeds arrive late or corrections are published. Training eligibility should use the availability rule that the deployed system could have followed. Preserve the original forecast as well. Replacing it with a newly generated path after the outcome arrives makes it harder to establish what the system actually predicted at the time. The [scikit-learn TimeSeriesSplit documentation](https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.TimeSeriesSplit.html) explains why time-ordered data needs a different evaluation split from methods that can train on the future and test on the past. Its gap parameter excludes samples at the end of a training set before the test set. For multi-step targets, I would inspect the actual target intervals around each boundary. A row issued before the validation period may have a label that uses observations inside it. Decide whether the experiment is a fixed training snapshot or a sequential replay, then enforce the corresponding label-availability rule. A numeric gap is useful only when its relationship to the sampling interval and target horizon is understood. The splitter also does not govern every transformation around it. Fit scalers, feature selection, and calibration on the permitted training data. Keep a record of which model and configuration each forecast used. For a pretrained model, investigate its training-data coverage and record uncertainty about overlap; a clean split in the local selector cannot establish that the foundation model never encountered the evaluation period. Traid's documented design starts with Ridge and promotes a gradient-boosted alternative only when chronological validation and evidence requirements support it. Keeping the simpler selector is an available outcome, not a failed experiment to hide. The case study reports improved candidate-quality scoring in its three-run research batch. It also explicitly says that a later cost-aware paper diagnostic did not establish a profitable strategy. These findings answer different questions. A better estimate of candidate quality does not itself establish useful execution, acceptable costs, or investment returns. I would retain unsuccessful experiments with their data boundaries and the reason they were rejected. Reusing those failures can make the next development iteration more informed. Repeatedly consulting a protected holdout to choose the next feature, however, spends that holdout's independence. Once it guides development, describe it as development evidence and arrange a fresh evaluation. The practical industry consequence extends beyond market research. Demand forecasts and other delayed-outcome systems face the same timing question: what could the model have learned when it made this decision? AI can shorten implementation and analysis. It cannot make an unfinished outcome arrive earlier. A useful research product makes that constraint visible, even when the next experiment is cheap to run.