# Purged and Embargoed Cross-Validation for Options ML

> Source: <https://dev.to/shaktitiwari/purged-and-embargoed-cross-validation-for-options-ml-46p0>
> Published: 2026-08-19 06:56:56+00:00

Why plain k-fold silently overfits your trading model — and the 4-line fix that stops it.

Financial data is sequential. k-fold shuffles rows, so a training row from 2 PM Tuesday sits

next to a test row from 10 AM Monday. Worse: **triple-barrier labels overlap**. A label at

bar *t* looks 6 bars into the future; a training row at *t+2* "knows" part of that future.

The model leaks.

V1's history is full of "HIGH overfit" verdicts — train AUC high, test AUC flat. Plain

`TimeSeriesSplit`

is only marginally better; it still lets adjacent windows bleed into each

other.

For each test window `[t0, t1]`

:

`max_training_horizon`

bars after the test window — drop those too.Overlapping labels are not i.i.d. Purging + embargoing makes the split honest.

``` python
def purged_embargo_split(n, n_splits=5, embargo_frac=0.02):
    idx = np.arange(n)
    fold = np.array_split(idx, n_splits)
    splits = []
    for i in range(n_splits):
        test = fold[i]
        emb = int(len(test) * embargo_frac)
        lo, hi = max(0, test[0]-emb), min(n, test[-1]+emb+1)
        train_mask = np.ones(n, bool); train_mask[lo:hi] = False
        splits.append((idx[train_mask], test))
    return splits
```

Optuna once "won" a validation set with only **4 decisive rows** — statistically meaningless.

Rule: never tune when the decisive (non-abstained) validation rows are below ~30–50. Widen the

date range or symbol basket first; don't trust the trial.

train (fit) → validation (early stop + HP select) → **disjoint calibration set** (sigmoid/

isotonic) → test (untouched, final score only). V1 sometimes conflated validation and

calibration. Keep them separate.

Log every trial's train/val/test gap, not just the winner's test score. Promote only if

replay AND shadow (≥1 live session) both beat baseline on **buyer metrics**: 1.5x/2.0x hit

rate, MAE-before-hit, time-to-hit, wrong-side rate.

*Research only. Not investment advice.*