{"slug": "protlingo-efficient-protein-language-modeling-via-conditional-memory-and-expert", "title": "ProtLingo: Efficient Protein Language Modeling via Conditional Memory and Expert Routing", "summary": "Researchers introduced ProtLingo, an efficient protein language model framework that augments a pretrained single-sequence backbone with conditional local memory and sparse expert routing, achieving competitive performance on protein fitness prediction, FLIP benchmarks, and supervised contact prediction with a 150M-scale backbone while improving parameter efficiency on mutation-effect prediction.", "body_md": "arXiv:2609.04793v1 Announce Type: new \nAbstract: Proteins perform diverse cellular functions, and even single amino-acid substitutions can alter stability, activity, or molecular interactions. Protein language models (PLMs) provide a scalable approach for modeling such sequence--function relationships from unlabeled sequences, but increasing the size of dense Transformer backbones often brings substantial computational cost without consistently improving mutation-sensitive prediction. We introduce ProtLingo, an efficient PLM framework that augments a pretrained single-sequence backbone with conditional local memory and sparse expert routing. ProtLingo maps contextual residue representations into route-specific discrete codes, composes centered local windows into latent $N$-gram addresses, and retrieves reusable residual signals associated with recurring local sequence contexts. In parallel, selected feed-forward blocks are upcycled into sparse Mixture-of-Experts layers with shared and routed experts, enabling residue-dependent computation while activating only a subset of parameters. Experiments on protein fitness prediction, FLIP benchmarks, and supervised contact prediction show that ProtLingo achieves competitive performance with a 150M-scale backbone, including strong parameter efficiency on mutation-effect prediction and preserved long-range structural representations.", "url": "https://wpnews.pro/news/protlingo-efficient-protein-language-modeling-via-conditional-memory-and-expert", "canonical_source": "https://www.machinebrief.com/news/protlingo-efficient-protein-language-modeling-via-conditiona-qxz0", "published_at": "2026-09-07 04:00:00+00:00", "updated_at": "2026-09-07 08:56:19.301669+00:00", "lang": "en", "topics": ["artificial-intelligence", "machine-learning", "large-language-models", "ai-research"], "entities": ["ProtLingo"], "alternates": {"html": "https://wpnews.pro/news/protlingo-efficient-protein-language-modeling-via-conditional-memory-and-expert", "markdown": "https://wpnews.pro/news/protlingo-efficient-protein-language-modeling-via-conditional-memory-and-expert.md", "text": "https://wpnews.pro/news/protlingo-efficient-protein-language-modeling-via-conditional-memory-and-expert.txt", "jsonld": "https://wpnews.pro/news/protlingo-efficient-protein-language-modeling-via-conditional-memory-and-expert.jsonld"}}