{"slug": "when-prediction-error-is-not-enough-evaluating-nuisance-function-prediction-for", "title": "When Prediction Error Is Not Enough: Evaluating Nuisance-Function Prediction for Causal Estimation", "summary": "A new arXiv preprint (arXiv:2609.00071v1) reports that prediction error is not a reliable proxy for causal estimator quality, based on Monte Carlo simulations comparing OLS, GAMs, XGBoost, and Double Machine Learning with XGBoost (DML-XGBoost) in a partially linear model. XGBoost achieved the lowest RMSE among non-oracle methods, while DML-XGBoost generally provided better 95% confidence interval coverage, but prediction error did not consistently track causal bias, and a joint-error measure was only weakly associated with causal bias.", "body_md": "arXiv:2609.00071v1 Announce Type: new\nAbstract: Prediction error is widely used to evaluate nuisance-function estimators in causal inference, but its relationship with causal estimator performance may differ across performance measures. We studied this question in a partially linear model using Monte Carlo simulations. We compared ordinary least squares (OLS), generalized additive models (GAMs), XGBoost, and Double Machine Learning with XGBoost (DML-XGBoost), evaluating nuisance-function prediction error, bias, RMSE, and 95\\% confidence interval coverage. We also examined a simple joint-error measure based on the absolute cross-product of estimation errors from the exposure and outcome nuisance functions. Across the simulated settings, XGBoost had the lowest RMSE among the non-oracle methods, while DML-XGBoost generally provided better confidence interval coverage. Prediction error did not consistently track causal bias across methods and settings, and the method with the best point-estimation performance did not necessarily have the best confidence interval coverage. The joint-error measure was only weakly associated with causal bias and did not provide a useful standalone measure of causal performance. These results suggest that prediction error is useful for assessing nuisance-function estimation, but it should not be treated as a direct measure of the quality of the resulting causal estimator.", "url": "https://wpnews.pro/news/when-prediction-error-is-not-enough-evaluating-nuisance-function-prediction-for", "canonical_source": "https://arxiv.org/abs/2609.00071", "published_at": "2026-09-02 04:00:00+00:00", "updated_at": "2026-09-02 04:26:37.349170+00:00", "lang": "en", "topics": ["machine-learning"], "entities": ["arXiv", "XGBoost", "Double Machine Learning"], "alternates": {"html": "https://wpnews.pro/news/when-prediction-error-is-not-enough-evaluating-nuisance-function-prediction-for", "markdown": "https://wpnews.pro/news/when-prediction-error-is-not-enough-evaluating-nuisance-function-prediction-for.md", "text": "https://wpnews.pro/news/when-prediction-error-is-not-enough-evaluating-nuisance-function-prediction-for.txt", "jsonld": "https://wpnews.pro/news/when-prediction-error-is-not-enough-evaluating-nuisance-function-prediction-for.jsonld"}}