{"slug": "riemannian-lorentz-fusion-of-vision-transformers-and-state-space-models", "title": "Riemannian--Lorentz Fusion of Vision Transformers and State-Space Models", "summary": "A new arXiv paper (2609.19384v1) proposes Riemannian–Lorentz Parameter Fusion (RLPF), a geometry-aware method for merging a Vision Transformer and a state-space model into a two-branch hybrid that reaches 82.37% on CIFAR-10, 75.04% on Oxford-IIIT Pet, and 78.58% top-1 accuracy on ImageNet-1K, versus best-parent accuracies of 76.54%, 71.42%, and 76.42%. RLPF projects semantically aligned parameter groups to common coordinates, lifts selected coordinates to the Lorentz hyperboloid model of hyperbolic space, computes a regularized geodesic barycenter, and decodes the result into both branches, with a learned gate combining branch logits per input. The authors report a pre-fine-tuning ImageNet-1K initialization of 77.80% and state the results support further study of geometry-aware heterogeneous fusion rather than a training-free single-checkpoint merge, since the gate and final models are trained.", "body_md": "arXiv:2609.19384v1 Announce Type: new \nAbstract: Scaling deep learning faces critical bottlenecks: data exhaustion, exponential training costs, and resource concentration. Model merging combines pre-trained checkpoints without gradient descent, offering orders-of-magnitude savings versus retraining. Combining independently trained vision models is difficult when their architectures and parameter shapes differ. Existing weight-space merging methods generally assume aligned, shape-compatible checkpoints, whereas a Vision Transformer (ViT) and a state-space model (SSM) implement token mixing with different operators. We study a hybrid Heterogeneous merging setting that retains both architectures while aligning parameter groups by semantic role. Our proposed Riemannian--Lorentz Parameter Fusion (RLPF) method projects aligned groups to common coordinates, lifts selected coordinates to the Lorentz hyperboloid model of hyperbolic space, computes a regularized geodesic barycenter, and decodes the result into the two branches. A learned gate then combines branch logits for each input. Component groups use fixed curvature values, with normalization parameters treated as Euclidean. In the results available in this manuscript, the fine-tuned system obtains 82.37\\% on CIFAR-10, 75.04\\% on Oxford-IIIT Pet, and 78.58\\% top-1 accuracy on ImageNet-1K; the corresponding best-parent accuracies are 76.54\\%, 71.42\\%, and 76.42\\%. On ImageNet-1K, the reported pre-fine-tuning initialization reaches 77.80\\%. These results support further study of geometry-aware heterogeneous fusion, but not a training-free single-checkpoint merge: RLPF is a two-branch hybrid whose gate and reported final models are trained.", "url": "https://wpnews.pro/news/riemannian-lorentz-fusion-of-vision-transformers-and-state-space-models", "canonical_source": "https://arxiv.org/abs/2609.19384", "published_at": "2026-09-18 04:00:00+00:00", "updated_at": "2026-09-18 04:27:25.472638+00:00", "lang": "en", "topics": ["computer-vision", "machine-learning", "ai-research", "neural-networks"], "entities": ["arXiv", "Riemannian–Lorentz Parameter Fusion", "Vision Transformer", "state-space model", "CIFAR-10", "Oxford-IIIT Pet", "ImageNet-1K", "Lorentz hyperboloid model"], "alternates": {"html": "https://wpnews.pro/news/riemannian-lorentz-fusion-of-vision-transformers-and-state-space-models", "markdown": "https://wpnews.pro/news/riemannian-lorentz-fusion-of-vision-transformers-and-state-space-models.md", "text": "https://wpnews.pro/news/riemannian-lorentz-fusion-of-vision-transformers-and-state-space-models.txt", "jsonld": "https://wpnews.pro/news/riemannian-lorentz-fusion-of-vision-transformers-and-state-space-models.jsonld"}}