{"slug": "when-does-pre-trained-capacity-help-causal-learning-from-scalar-prediction-to", "title": "When Does Pre-Trained Capacity Help Causal Learning? From Scalar Prediction to Counterfactual Post-Training", "summary": "A study of two systems built on the Qwen 2.5 family found that scaling the backbone from 0.5B to 1.5B parameters produced mixed scalar-estimation results on IHDP but raised CodeGen pass@1 from 0.267 to 0.575 and treatment-effect score from 0.231 to 0.515 after the full generative pipeline, according to arXiv paper 2610.33311v1. Within the 0.5B CausalLLM system, narrative inputs cut mean sqrt(PEHE) from 1.842 to 1.780, a 3.4% gain over tabular inputs, and the 5.8% improvement over a no-LLM baseline combined representation and input-format changes. The authors caution that the two systems differ in dataset, architecture, output modality and supervision, with intermediate-stage evaluations unavailable, so the results with two small backbones and three replications support a hypothesis about capacity-adaptation alignment rather than a general scaling law.", "body_md": "arXiv:2610.33311v1 Announce Type: new \nAbstract: How does pre-trained model capacity relate to performance after adaptation for causal tasks? We study two systems built on the Qwen 2.5 family: CausalLLM, which estimates scalar potential outcomes with treatment-specific heads, and CausalGen, which generates treatment-conditioned text or code using soft prefixes and factual supervised fine-tuning (SFT), counterfactual SFT, and preference alignment. Increasing backbone size from 0.5B to 1.5B yields mixed scalar-estimation results on IHDP, but improves CodeGen pass@1 from 0.267 to 0.575 and treatment-effect score from 0.231 to 0.515 after the complete generative pipeline. Within 0.5B CausalLLM, narrative inputs reduce mean sqrt(PEHE) from 1.842 to 1.780, a 3.4% improvement over tabular inputs; the 5.8% improvement over a no-LLM baseline combines changes in representation and input format. These descriptive comparisons motivate a distinction between available and convertible capacity. They do not isolate the contribution of post-training: the two systems differ in dataset, architecture, output modality, and supervision, and intermediate-stage evaluations are unavailable. With only two small backbones and three replications, the results support a hypothesis about capacity--adaptation alignment rather than a general scaling law or a demonstrated causal mechanism.", "url": "https://wpnews.pro/news/when-does-pre-trained-capacity-help-causal-learning-from-scalar-prediction-to", "canonical_source": "https://arxiv.org/abs/2610.33311", "published_at": "2026-10-28 04:00:00+00:00", "updated_at": "2026-10-28 04:14:06.791770+00:00", "lang": "en", "topics": ["artificial-intelligence", "machine-learning", "large-language-models", "ai-research"], "entities": ["Qwen 2.5", "CausalLLM", "CausalGen", "IHDP", "arXiv"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/when-does-pre-trained-capacity-help-causal-learning-from-scalar-prediction-to", "markdown": "https://wpnews.pro/news/when-does-pre-trained-capacity-help-causal-learning-from-scalar-prediction-to.md", "text": "https://wpnews.pro/news/when-does-pre-trained-capacity-help-causal-learning-from-scalar-prediction-to.txt", "jsonld": "https://wpnews.pro/news/when-does-pre-trained-capacity-help-causal-learning-from-scalar-prediction-to.jsonld"}}