11:46
2026-08-23
discuss.huggingface.co
artificial-intelligence
I built an experimental routing-based attention mechanism for GPT models
Noah DeAngelis released RoutingGPT, an experimental GPT language model built in PyTorch that uses a routing-based attention mechanism to reduce computation from TĆT to TĆ0.25T, aiming to make longer cā¦