Reward-Informed Sparse Autoencoders and the Solution-Completeness Confound

A new study from arXiv (2608.26136v1) finds that reward-informed sparse autoencoders (RI-SAEs) trained on Llama-3.1-8B separate high- and low-reward reasoning traces mainly by solution completeness, not reasoning quality: a TF-IDF text classifier splits the classes with AUC 0.75–0.83, and three structural cues (length, closed reasoning block, boxed answer) reach AUC 0.70, with 99% of good versus 69% of bad completions boxed. The authors present the RI-SAE recipe and a control battery, noting that a reward-blind SAE shows no class separation (silhouette 0.01), while the reward-informed SAE's selected features reach silhouette 0.79, and only two discriminative features are readable (symbolic mathematics; procedural and evaluative language).

arXiv:2608.26136v1 Announce Type: new Abstract: Sparse autoencoders SAEs decompose language-model activations into sparse, interpretable features, and an appealing way to aim them at reasoning is to curate their data with a signal reinforcement learning already produces: the reward. We build such a reward-informed SAE RI-SAE : we split GRPO trajectories into high-reward "good" and low-reward "bad" reasoning continuations, train a standard JumpReLU SAE on their activations, and then ask what the resulting good/bad separation actually measures. On Llama-3.1-8B a sparse subset of the 16,384 features does separate the classes silhouette 0.79 on the selected features versus 0.005 for the full code , but a control battery shows the separation is largely solution completeness rather than reasoning quality: a TF-IDF text classifier already splits the classes AUC 0.75--0.83 , and three structural cues alone length, a closed reasoning block, and a boxed answer reach AUC 0.70 99% of good versus 69% of bad completions are boxed . A generic SAE that never saw the reward does not separate the classes at all silhouette 0.01, no discriminative features , so the 0.79 is in-sample fitting of this curated signal rather than structure that a reward-blind dictionary recovers. We therefore present the recipe and its control battery together: reward filtering is a cheap, label-free way to reuse RL signals for interpretability, but most of what it surfaces is completion form. Two discriminative features are still readable symbolic mathematics; procedural and evaluative language , which we take as illustrative rather than as isolated reasoning.