{"slug": "accurate-ensembles-fragile-narratives-multi-scale-stacking-and-a-fidelity-audit", "title": "Accurate Ensembles, Fragile Narratives: Multi-Scale Stacking and a Fidelity Audit of LLM-Generated Explanations for Credit Risk", "summary": "A multi-scale stacking ensemble for credit risk scoring reached test ROC-AUC 0.9539 and PR-AUC 0.9137 on a 32,581-application dataset, beating the best single model by Delta-AUC 0.0143, but an audit of LLM-generated explanations found the model named three factors as risk-increasing that attributions scored as risk-reducing, omitted the dominant driver, and introduced a feature never given to it, leading researchers to conclude that grounding must be verified after generation, not assumed.", "body_md": "arXiv:2608.08126v1 Announce Type: new\nAbstract: Credit scoring increasingly relies on models whose decision logic cannot be read off their parameters, in tension with supervisory expectations that adverse decisions be explainable. A common proposal closes that gap with a language model: compute feature attributions, hand them to an LLM, and let it write the rationale. We build such a system end to end and test whether the second half of the promise holds. The predictive component is a multi-scale stacking ensemble fusing four differently regularised gradient-boosting learners with a residual network through a neural meta-learner trained on out-of-fold predictions. On a public 32,581-application credit dataset it reaches test ROC-AUC 0.9539 (95% CI [0.9462, 0.9616]) and PR-AUC 0.9137, beating the best single model by Delta-AUC = 0.0143 (p = 0.016 under a conservative independence assumption). Our central finding is asymmetric. The ranking gain is real but operationally small: at the F1-optimal threshold the ensemble avoids only six additional missed defaults out of 1,422 against a tuned random forest, cutting cost-weighted loss by under 2%. The narrative layer fails in a way prompt engineering alone does not fix. In an audited case the model named three factors as risk-increasing that the supplied attributions scored as risk-reducing, omitted the dominant driver, and introduced a feature never given to it. We trace this to properties we measure rather than assume: SHAP and LIME agree on which features matter (overlap@10 = 0.80) but not on their order (tau = 0.43, p = 0.18), and the attribution sign for the model's most sensitive input is near a coin flip across applicants (modal-sign share 0.53). Calibration (ECS = 0.117) and perturbation stability (DPD = 0.078) both fall short of our own thresholds. Constrained prompting is necessary but not sufficient: grounding must be verified after generation, not assumed.", "url": "https://wpnews.pro/news/accurate-ensembles-fragile-narratives-multi-scale-stacking-and-a-fidelity-audit", "canonical_source": "https://www.machinebrief.com/news/accurate-ensembles-fragile-narratives-multi-scale-stacking-a-yd0h", "published_at": "2026-08-11 04:00:00+00:00", "updated_at": "2026-08-11 05:13:08.356077+00:00", "lang": "en", "topics": ["machine-learning", "artificial-intelligence", "large-language-models", "ai-ethics"], "entities": ["arXiv", "SHAP", "LIME"], "alternates": {"html": "https://wpnews.pro/news/accurate-ensembles-fragile-narratives-multi-scale-stacking-and-a-fidelity-audit", "markdown": "https://wpnews.pro/news/accurate-ensembles-fragile-narratives-multi-scale-stacking-and-a-fidelity-audit.md", "text": "https://wpnews.pro/news/accurate-ensembles-fragile-narratives-multi-scale-stacking-and-a-fidelity-audit.txt", "jsonld": "https://wpnews.pro/news/accurate-ensembles-fragile-narratives-multi-scale-stacking-and-a-fidelity-audit.jsonld"}}