02:58
2026-07-16
discuss.huggingface.co
machine-learning
My latest ablation run: integrating Engram onto two backbones
A 200-step, ~1.7B-parameter ablation run in OLMo-core comparing Engram on a standard attention Transformer versus a 3 GDN-layer + 1 attention-layer hybrid found that Transformer + Engram reached sligh…