arXiv:2611.25167v1 Announce Type: new Abstract: Can the allocation used to train experts also predict which expert will perform best on a prompt? Such a connection would make the intended division of labor inspectable before training and allow the allocation itself to guide inference. We introduce InflEns, a mixture of LoRA experts whose training weights and inference routes come from the same location game in a fixed prompt-trait space. Players choose positions to maximize their expected shares of the prompt distribution, without using expert losses. We utilize the game's kernel-general theoretical criterion for when co-located positions become locally unstable to separation. We evaluate InflEns on a corpus assembled from seven safety-related benchmarks, which demonstrates that the highest-share expert has the lowest held-out response negative log-likelihood (NLL) on approximately 90% of prompts across four kernels under a sharpened simplex representation, compared with approximately 15% after permuting allocations across prompts. We demonstrate that the method performs comparably to published learned gates without having to train an explicit gate on the experts. Because the router is fixed, we can trace harmful behavior to a single expert and retrain only that expert, which lowers HarmBench attack success from 0.51 to 0.02 while keeping instruction following. The game also gives us several controls over how the experts specialize, including the kernel choice and its parameters, the trait-space geometry, and the number of experts. Importantly, InflEns is a general framework that can be extended to any encoder and data distribution.
Game-Theoretic Data Allocation for Expert Specialization and Routing
A new arXiv paper (2611.25167v1) introduces InflEns, a mixture-of-LoRA-experts method in which both training weights and inference routing derive from a single location game in a fixed prompt-trait space, so the highest-share expert had the lowest held-out response negative log-likelihood on approximately 90% of prompts across four kernels, versus approximately 15% after permuting allocations across prompts. Evaluated on a corpus assembled from seven safety-related benchmarks, InflEns performed comparably to published learned gates without training an explicit gate, and because the router is fixed the authors traced harmful behavior to a single expert and retrained only that expert, lowering HarmBench attack success from 0.51 to 0.02 while keeping instruction following. The authors describe InflEns as a general framework extendable to any encoder and data distribution.
When safety failures trace to one expert in a mixture, retrain only that expert rather than the whole model; the authors cut HarmBench attack success from 0.51 to 0.02 while keeping instruction following.
more in playbook: Fine-tuning and dataRun your AI side-project on zahid.host
EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.