Zebra-HyLo: Upcycling Transformers into Long-Context Hybrid LLMs on AMD Instinct™ GPUs
AMD researchers introduced Zebra-HyLo, an upcycling recipe that converts pretrained Transformer models into long-context hybrid LLMs through post-training on roughly 10B tokens, without pretraining from scratch. The meth…