Towards Scalable Customization and Deployment of Multi-Agent Systems for Enterprise Applications

Researchers propose a unified framework for customizing and deploying LLM-based multi-agent systems in enterprise settings, combining model adaptation techniques with inference optimizations that achieve a 4.48x throughput speedup while maintaining performance.

arXiv:2606.18502v1 Announce Type: new Abstract: Large language model LLM -based multi-agent systems demonstrate strong performance on complex reasoning and task execution, enabling broad enterprise applications. However, production deployment remains challenging due to domain-specific customization requirements and high latency and inference costs in agentic workflows. We propose a unified framework for customization and efficient deployment of multi-agent systems in real-world settings. The first stage, Agentic Model Customization, combines continual pretraining, supervised fine-tuning, and preference optimization to adapt a compact model to specialized domains while retaining strong agentic capabilities. The second stage, Inference Optimization, integrates speculative decoding and FP8 quantization with targeted calibration to enable cost-efficient serving with minimal quality loss. Across enterprise workloads, our framework enables rapid domain adaptation and achieves a 4.48x speedup in throughput while maintaining performance and improving robustness on long-tail scenarios.