Auditability Is Not One Property: Rule Overlap, Behavioural Agreement, and Composition in Reinforcement Learning

A new arXiv paper (2609.28581v1) reports that rule-set overlap between independently trained reinforcement-learning policies does not imply behavioral agreement, with policies sharing symbolic rules while choosing near-chance-matching actions on fresh states. The authors define auditability as six separately testable predicates — trace integrity, lossless coding, rule coverage, behavioral agreement, composition quality, and value-model reliability — and use a shared frozen symbolizer, passive rule extraction, an append-only hash-bound ledger, exact environment replay, and offline confidence-ranked arbitration with an explicit blind-spot fallback. On a conflict-dominated task, an apparent fusion failure was traced to an induction/deployment mismatch, where rules induced from sampled actions were evaluated under argmax actions and deployment-consistent re-induction reversed the arbitration ordering; a fitted-Q generalized-policy-improvement diagnostic also failed in both environments.

arXiv:2609.28581v1 Announce Type: new Abstract: Reinforcement-learning RL policies are often distributed as opaque neural checkpoints, while training logs show that a run occurred without explaining what the policy learned. We study whether independently trained policies can be represented and composed through auditable discrete behavioral rules. We define auditability as six separately testable predicates: trace integrity, lossless coding, rule coverage, behavioral agreement, composition quality, and value-model reliability. Our protocol uses a shared frozen symbolizer, passive rule extraction, an append-only hash-bound ledger, exact environment replay, and offline confidence-ranked arbitration with an explicit blind-spot fallback. The results place strict limits on this description layer. Rule-set overlap does not imply behavioral agreement: policies may share symbolic rules while choosing near-chance-matching actions on fresh states. The fused policy therefore selects among existing rules rather than generating a new skill. On a conflict-dominated task, an apparent fusion failure is traced to an induction/deployment mismatch: rules induced from sampled actions were evaluated under argmax actions, and deployment-consistent re-induction reverses the arbitration ordering. A fitted-Q generalized-policy-improvement diagnostic also fails in both environments, limiting claims that rule fusion is superior to value-based composition. One exploratory comparison favors rule fusion, but its comparator is post hoc, the task is partly saturated, and the fused policy remains below the strongest held-out actor. We contribute an evidence-bounded audit and composition protocol, not a claim of universal interpretability or autonomous skill generation. Future work must add temporally extended skills, cross-skill interfaces, composition search, and independent novelty audits.