A Bellman Optimality Equation for Plasticity

A new arXiv paper (arXiv:2609.10776v1) presents preliminary work showing that a Bellman optimality equation exists for optimizing plasticity within Markov decision processes, building on Abel et al. (2025), which defined plasticity as the generalized directed information from an agent's observations to its actions and empowerment as the generalized directed information from its actions to its observations. The paper addresses the previously unstudied problem of optimizing plasticity under that definition, reframing the traditional stability-plasticity tradeoff in continual reinforcement learning as an empowerment-plasticity tradeoff.

arXiv:2609.10776v1 Announce Type: new Abstract: In continual reinforcement learning, carefully managing the stability-plasticity tradeoff remains a core challenge. Recent work by Abel et al. 2025 formalized this dilemma by defining plasticity as the generalized directed information from an agent's observations to its actions, and empowerment as the generalized directed information from its actions to its observations. This formulation successfully reframes the traditional stability-plasticity tradeoff as an empowerment-plasticity tradeoff. However, while extensive literature exists on optimizing for empowerment, there is currently no research addressing the optimization of plasticity under this new definition. This paper presents preliminary work toward optimizing plasticity within Markov decision processes. We show that there exists a Bellman optimality equation for optimizing plasticity similar to previous work for empowerment.