Lessons from a real rebuild of an options-buyer prediction system. No profit claims —
just the architecture that fixes the chronic bugs of V1.
V1 asked one XGBoost model one big fuzzy question: "CE ya PE?" — directly from raw
CE/PE premium data. Premium is a transformed signal (underlying move × delta × gamma × IV ×
theta × spread × strike distance × liquidity). The model learned noise as much as signal.
Concrete evidence from the research logs:
lr=0.02, depth=3
defaults used throughout; Optuna existed but was never run).iv_change_1d
shift inside single-row groups) silently zeroed a whole feature for the entire history.
underlying mechanics --> side, range, ETA, invalidation
option chain scanner --> is the buyer contract worth paying for?
XGBoost (many heads) --> thin calibrated learner on clean mechanics
Rule: underlying decides side; option contract decides execution eligibility. CE/PE
premium is validated against, never learned as, direction.
Instead of one CE/PE answer, V2 trains separate narrow heads:
underlying_up/down_touch_{15,30,60}m
ce_1p3x / ce_1p5x / ce_2p0x
and pe_1p3x / pe_1p5x / pe_2p0x
(SEPARATE CE and PE)no_trade_quality
This single change removes most of the CE/PE confusion V1 fought for months.
learning_rate = 0.015–0.035 n_estimators = 800–2000 (early stop)
max_depth = 2–3 min_child_weight = 12–40
gamma = 0.1–2.0 subsample = 0.65–0.90
colsample_bytree = 0.55–0.85 reg_alpha = 0.5–3.0
reg_lambda = 6.0–20.0 scale_pos_weight = min(neg/pos, 8.0)
V1's intraday head had only 8 of 1280 features with non-zero gain — most of the bloat
was pure noise the regularizer had to prune. Shallow + hard-regularized is the answer.
overfit_gap = train_metric − test_metric
. Flag if > 0.15. A model is NOT promoted just
because train metrics look good. Log the gap automatically on every head, every retrain.
V2 is a cleaner architecture, but it is still research. The lesson that transfers: stop
asking fuzzy questions, declare your nulls, keep trees shallow, and gate promotion on
out-of-sample gap — not training score.
Research only. Not investment advice.