From Residuals to Reasons: LLM-Guided Mechanism Inference from Tabular Data

Researchers have developed Multi-Agent Residual In-Context Learning (MARICL), a framework that uses large language model agents to analyze where a base statistical model fails and generate explicit correction terms from high-residual examples. Tested across nine scientific and socioeconomic benchmarks, MARICL consistently improved predictions, and frozen formulas from one experimental batch of the Cell-Free Protein dataset improved predictions in over 92% of held-out batches under the same protocol. The framework's success boundary aligned with underlying biochemistry rather than batch identity, providing direct evidence of mechanistic generalization rather than noise fitting.

arXiv:2605.22897v1 Announce Type: new Abstract: A persistent challenge in machine learning for scientific applications is jointly achieving prediction and understanding. Statistical models excel on structured data but operate as black boxes, while existing interpretability methods are largely inspective: they answer "which features matter?" but do not articulate how features interact or refine explanations iteratively alongside human understanding. Asking an LLM to predict the target directly forces it to search the entire output space; we instead anchor predictions with a base model and ask the LLM the narrower question of what that model is missing. We introduce Multi-Agent Residual In-Context Learning MARICL , an agentic framework in which LLM agents analyze where a base-model fails, hypothesize missing structure from high-residual examples provided in context, and produce explicit correction terms refined through multi-turn textual gradient optimization. Across nine benchmarks spanning scientific, biomedical, socioeconomic, and synthetic settings, MARICL improves consistently over its base model on all datasets. To test whether these corrections reflect real structure or batch-specific noise, we freeze formulas learned on one experimental batch of the Cell-Free Protein dataset and apply them with no retraining and no further LLM calls to held-out batches. Within the same reagent protocol, the frozen formulas improve predictions in over 92% of cases; across a different protocol, they fail systematically. The success boundary aligns with the biochemistry, not the batch count; direct evidence of mechanistic generalization.