{"slug": "metaencoder-exploring-the-limit-of-bi-encoders-for-multimodal-system-one-making", "title": "MetaEncoder: Exploring the Limit of Bi-Encoders for Multimodal System One Decision Making with Natural Language Interface", "summary": "MetaEncoder, a multimodal encoder fine-tuned from a Muse-Glimmer decoder via unidirectional contrastive learning, outperforms state-of-the-art multimodal embedding baselines on decision tasks and on MMEB-v3 image and video retrieval, according to an arXiv paper (2610.11316v2). The authors' comparison of bi-encoders, cross-encoders and constrained decoders found cross-encoders consistently achieve the highest accuracy across model sizes, while bi-encoders keep near-constant inference latency as candidate pools grow because candidate embeddings can be cached offline, whereas cross-encoder and constrained-decoder costs scale quadratically with candidate size. Using packed contrastive training, the team memory-efficiently full fine-tuned a 30B decoder into an encoder and evaluated MetaEncoder across 12 benchmark suites spanning closed-set decision-making, multimodal understanding and open-set retrieval.", "body_md": "arXiv:2610.11316v2 Announce Type: replace \nAbstract: System One models prioritize fast, low-latency decision-making and calibrated probability distributions over token generation. While prevailing systems rely on structured schemas and typed outputs, candidate-scoring trade-offs remain poorly understood. We systematically investigate the scaling dynamics, accuracy, and inference efficiency of three scoring architectures adapted from a pre-trained decoder: bi-encoders, cross-encoders, and constrained decoders. Experiments show clear performance scaling with model size across all three architectures. Cross-encoders consistently achieve the highest accuracy across model sizes, while bi-encoders and constrained decoders perform similarly as training tokens scale. Efficiency-wise, bi-encoders maintain near-constant inference latency as candidate pools grow because candidate embeddings can be cached offline, whereas cross-encoder and constrained-decoder costs scale quadratically with candidate size. This efficiency advantage makes bi-encoders uniquely suited to bridge small closed-set decisions and massive open-set retrieval. For closed-set tasks, including all candidate options in the request prompt is critical for bi-encoder competitiveness. Guided by these insights, we introduce MetaEncoder, a multimodal encoder fine-tuned from a Muse-Glimmer decoder via unidirectional contrastive learning to match requests and candidates. We present packed contrastive training to maximize efficiency for dynamically allocated negative candidates, enabling memory-efficient full fine-tuning of a 30B decoder into an encoder. Evaluated across 12 benchmark suites spanning closed-set decision-making, multimodal understanding, and open-set retrieval, MetaEncoder outperforms state-of-the-art multimodal embedding baselines on decision tasks and MMEB-v3 image/video retrieval, while scaling to open candidate spaces far too large for current decision models.", "url": "https://wpnews.pro/news/metaencoder-exploring-the-limit-of-bi-encoders-for-multimodal-system-one-making", "canonical_source": "https://www.machinebrief.com/news/metaencoder-exploring-the-limit-of-bi-encoders-for-multimoda-r06z", "published_at": "2026-11-09 05:00:00+00:00", "updated_at": "2026-11-09 05:23:20.373291+00:00", "lang": "en", "topics": ["machine-learning", "ai-research", "large-language-models", "natural-language-processing", "artificial-intelligence"], "entities": ["MetaEncoder", "Muse-Glimmer", "MMEB-v3", "arXiv"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/metaencoder-exploring-the-limit-of-bi-encoders-for-multimodal-system-one-making", "markdown": "https://wpnews.pro/news/metaencoder-exploring-the-limit-of-bi-encoders-for-multimodal-system-one-making.md", "text": "https://wpnews.pro/news/metaencoder-exploring-the-limit-of-bi-encoders-for-multimodal-system-one-making.txt", "jsonld": "https://wpnews.pro/news/metaencoder-exploring-the-limit-of-bi-encoders-for-multimodal-system-one-making.jsonld"}}