DAIS: Dependency-Aware Intermediate QA Supervision for Complex Reasoning

A new training-time framework called Dependency-Aware Intermediate QA Supervision (DAIS) improves final-answer accuracy on complex reasoning tasks by converting filtered teacher rationales into stage-level QA records that condition each intermediate answer on previous states. Across GDPR, AIACT, MedQA, and FOLIO benchmarks with multiple Qwen backbones, DAIS achieves an average gain of 4.2% and a largest gain of 5.6% over the strongest non-DAIS baseline, outperforming answer-only, flat chain-of-thought, and independent-QA approaches.

arXiv:2607.19088v1 Announce Type: new Abstract: Chain-of-thought CoT supervision exposes intermediate rationales, but flat rationale targets usually optimize a single reasoning sequence and provide limited supervision on how local conclusions should support later decisions. We introduce Dependency-Aware Intermediate QA Supervision DAIS , a training-time framework that converts filtered teacher rationales into stage-level QA records. Each intermediate record predicts a local answer conditioned on the previous states needed for that decision, while the final-answer record keeps the original task format; evaluation therefore uses only the original input and optional context. Across GDPR, AIACT, MedQA, and FOLIO with multiple Qwen backbones, DAIS improves average final-answer accuracy over answer-only, flat chain-of-thought, and independent-QA baselines. On policy-compliance benchmarks, it achieves a largest gain of 5.6% and an average gain of 4.2% over the strongest non-DAIS baseline. Controlled ablations show that valid previous-state conditioning contributes beyond longer targets or additional intermediate text, supporting dependency-conditioned intermediate QA as a lightweight auxiliary supervision signal for standard final-answer inference.