Modular Deep Recurrent Neural Network: Application to Quadrotors

Researchers introduced a modular deep Recurrent Neural Network (RNN) architecture that adds feedforward inter-layer connections to improve learning of high-order dynamics and nonlinearities, alleviating vanishing/exploding gradients in multilayer RNNs. In a quadrotor case study, the new network learned altitude dynamics where existing methods failed to generalize, as reported in arXiv:2609.04339v1.

arXiv:2609.04339v1 Announce Type: new Abstract: A modular deep Recurrent Neural Network RNN is introduced to facilitate the process of deploying various architectures of RNNs, and to automatically compute derivatives for gradient-based learning methods. The modularity leads to a set of new architectures, one of which includes feedforward inter-layer connections. By adding feedforward inter-layer connections in a multi-layer RNN, it is observed that the capability of the RNN to learn and model high-order dynamics and nonlinearities is significantly improved. The problem of vanishing/exploding gradient in space for a multilayer RNN is also alleviated using feedforward connections. These results are demonstrated using a quadrotor case study, for which a model of the altitude dynamics is learned with our particular network structure, while existing methods are unable to generalize as quickly or at all.