LSTM-UT and Recurrent-Depth Transformers on Cellular Automata
A new LSTM Universal Transformer (LSTM-UT) with bounded gated memory outperformed both the Block Universal Transformer (BUT) and CoTFormer on Rule 30 cellular automata and a delayed-recall task, according to an arXiv pap…