Beyond Outcome Gaps: Process-Aware Fairness Diagnosis for LLM-based Multi-Agent Decision Systems

A new arXiv preprint (2609.02092v1) introduces SCOPED-Hiring, a process-aware fairness diagnosis pipeline for LLM-based hiring multi-agent systems, which logged over 311,000 structured decision trajectories and found that balanced final hire rates can mask hidden trajectory unfairness, such as career gaps triggering suspicion and identity cues leading to unequal investigation. Targeted repair guided by these diagnoses reduced total layered burden by 72.3% while shifting the hire rate by only 1.86 percentage points, demonstrating that process diagnosis can guide effective repair.

arXiv:2609.02092v1 Announce Type: new Abstract: LLM-based multi-agent systems MAS are increasingly considered for high-stakes decision-making, yet outcome-based fairness audits can miss where risks arise within the decision trajectory. We present SCOPED-Hiring, a process-aware fairness diagnosis pipeline for LLM-based hiring MAS. SCOPED-Hiring constructs controlled resume variants, runs role-based hiring committees, logs over 311K structured decision trajectories, and converts trajectory fields into quantitative fairness signals organized by six diagnostic lenses: final outcome, counterfactual, process, pathway, dynamic, and design effects. SCOPED-Hiring reveals that balanced final hire rates can mask hidden trajectory unfairness in multi-agent decision trajectories: career gaps trigger suspicion, proxy cues shape qualification judgments, and identity cues lead to unequal investigation. Targeted repair guided by these diagnoses reduces total layered burden by 72.3% while shifting the hire rate by only 1.86 pp, showing that process diagnosis can guide effective repair. Project Page: https://scoped-hiring-project-page.vercel.app/