OrchSLM: Probing the Dynamics of Small Language Model Orchestration

Researchers introduced OrchSLM, a routing framework that unifies existing non-interactive small language model (SLM) orchestration methods and exposes their design choices as controllable parameters, according to the arXiv paper 2609.13470v1. OrchSLM lets heterogeneous SLMs independently generate candidate solutions while a router orchestrates their cached samples without further model interaction, addressing the latency, privacy, connectivity, and computational cost limits of cloud-scale large language models in agentic pipelines. The framework serves as a systematic probe revealing how orchestration behavior emerges from task structure, model-pool composition, and multi-agent consensus.

arXiv:2609.13470v1 Announce Type: new Abstract: Although large language models LLMs have demonstrated remarkable capabilities, their reliance on cloud-scale infrastructure poses fundamental challenges for deployment in agentic pipelines, including latency, privacy, connectivity, and substantial computational cost. Small language models SLMs offer a compelling alternative: recent studies suggest that many repetitive and narrowly scoped subtasks in agentic workloads may be better served by specialized SLMs than by monolithic LLMs. However, the limited capacity and context windows of SLMs can constrain long-horizon reasoning and interaction-heavy orchestration strategies such as iterative verification and debate. This motivates a complementary, non-interactive paradigm in which heterogeneous SLMs independently generate candidate solutions and a router orchestrates their cached samples without further model interaction. To further understand the mechanisms of such orchestration, we introduce OrchSLM, a routing framework that unifies existing non-interactive orchestration methods and exposes their underlying design choices as controllable parameters. Using OrchSLM as a systematic probe, we reveal how orchestration behavior emerges from diverse knobs, including the task structure, model-pool composition, and multi-agent consensus.