{"slug": "purged-and-embargoed-cross-validation-for-options-ml", "title": "Purged and Embargoed Cross-Validation for Options ML", "summary": "A developer detailed how plain k-fold cross-validation silently overfits options trading models due to sequential data and overlapping triple-barrier labels, and proposed a purged and embargoed split to make validation honest. The fix drops training rows near test windows to prevent leakage, and the developer advises against tuning when decisive validation rows are below 30-50, emphasizing separate calibration sets and buyer metrics for model promotion.", "body_md": "Why plain k-fold silently overfits your trading model — and the 4-line fix that stops it.\n\nFinancial data is sequential. k-fold shuffles rows, so a training row from 2 PM Tuesday sits\n\nnext to a test row from 10 AM Monday. Worse: **triple-barrier labels overlap**. A label at\n\nbar *t* looks 6 bars into the future; a training row at *t+2* \"knows\" part of that future.\n\nThe model leaks.\n\nV1's history is full of \"HIGH overfit\" verdicts — train AUC high, test AUC flat. Plain\n\n`TimeSeriesSplit`\n\nis only marginally better; it still lets adjacent windows bleed into each\n\nother.\n\nFor each test window `[t0, t1]`\n\n:\n\n`max_training_horizon`\n\nbars after the test window — drop those too.Overlapping labels are not i.i.d. Purging + embargoing makes the split honest.\n\n``` python\ndef purged_embargo_split(n, n_splits=5, embargo_frac=0.02):\n    idx = np.arange(n)\n    fold = np.array_split(idx, n_splits)\n    splits = []\n    for i in range(n_splits):\n        test = fold[i]\n        emb = int(len(test) * embargo_frac)\n        lo, hi = max(0, test[0]-emb), min(n, test[-1]+emb+1)\n        train_mask = np.ones(n, bool); train_mask[lo:hi] = False\n        splits.append((idx[train_mask], test))\n    return splits\n```\n\nOptuna once \"won\" a validation set with only **4 decisive rows** — statistically meaningless.\n\nRule: never tune when the decisive (non-abstained) validation rows are below ~30–50. Widen the\n\ndate range or symbol basket first; don't trust the trial.\n\ntrain (fit) → validation (early stop + HP select) → **disjoint calibration set** (sigmoid/\n\nisotonic) → test (untouched, final score only). V1 sometimes conflated validation and\n\ncalibration. Keep them separate.\n\nLog every trial's train/val/test gap, not just the winner's test score. Promote only if\n\nreplay AND shadow (≥1 live session) both beat baseline on **buyer metrics**: 1.5x/2.0x hit\n\nrate, MAE-before-hit, time-to-hit, wrong-side rate.\n\n*Research only. Not investment advice.*", "url": "https://wpnews.pro/news/purged-and-embargoed-cross-validation-for-options-ml", "canonical_source": "https://dev.to/shaktitiwari/purged-and-embargoed-cross-validation-for-options-ml-46p0", "published_at": "2026-08-19 06:56:56+00:00", "updated_at": "2026-08-19 07:12:20.281626+00:00", "lang": "en", "topics": ["machine-learning", "mlops"], "entities": ["Optuna", "V1"], "alternates": {"html": "https://wpnews.pro/news/purged-and-embargoed-cross-validation-for-options-ml", "markdown": "https://wpnews.pro/news/purged-and-embargoed-cross-validation-for-options-ml.md", "text": "https://wpnews.pro/news/purged-and-embargoed-cross-validation-for-options-ml.txt", "jsonld": "https://wpnews.pro/news/purged-and-embargoed-cross-validation-for-options-ml.jsonld"}}