A Transdiagnostic Space of Disorder Like Phenotypes in Reinforcement Learning Agents

Researchers modeled seven psychological disorders in reinforcement learning agents by manipulating cognitive appraisal signals, revealing a two-dimensional affective space where disorders self-organize and interact nonadditively. The framework, tested across over a thousand runs and two environments, offers a controllable space for studying disorder induction and treatment.

arXiv:2607.07753v1 Announce Type: new Abstract: Modelling psychological disorders in artificial agents offers both a testbed for computational psychiatry and a lens on the failure modes of affective control. Prior work induces one or two disorders in a reinforcement learning RL agent by hand-tuned reward shaping, labels the behaviour post hoc, and reports single runs. We recast disorder modelling as dose-controllable manipulation of cognitive appraisal signals in an appraisal-guided PPO agent, expressing seven disorders anxiety, mania, obsessive-compulsive checking, depression, impulsivity, addiction, and post-traumatic stress each as a single knob grounded in a computational psychiatry account, with each symptom measured by a preregistered assay mapped to a recognised paradigm. Across more than a thousand runs 10 seeds, four controls, 95% confidence intervals every disorder shows a graded, monotone dose-response that no control reproduces. Beyond these induced effects, three findings emerge that were not written into the reward: the disorders self-organise into a two-dimensional affective space in which mania mirrors anxiety; removing a knob remits reward distortion disorders mania, checking, addiction but not avoidance disorders anxiety, PTSD , which instead recover under a graded exposure curriculum; and two simultaneous knobs interact nonadditively, yielding testable comorbidity predictions. Appraisal weights thus parameterise a controllable space of affective phenotypes in which the same knobs that induce a disorder can model its treatment. We also show that three disorder knobs depression, addiction, anxiety transfer to a three-dimensional pixel environment MiniWorld with a standard convolutional agent and no appraisal critic, with cross-assay dissociation confirmed across both domains, indicating the framework is not specific to grid worlds or to PPO's appraisal critic.