{"slug": "speculative-macro-commit-for-faster-tool-using-agents", "title": "Speculative Macro Commit for Faster Tool-Using Agents", "summary": "Researchers introduced Speculative Macro Commit (SMC), a runtime mechanism that reduces latency for tool-using LLM agents by having a faster speculative drafter model pre-execute action chains on an environment snapshot, which are committed when they match the authoritative actor's next action. Using Qwen3.5-27B INT4 as the actor and Qwen3.5-4B as the drafter, SMC cut latency by 10.23% over the Speculative Actions baseline and 18.59% over sequential execution on the τ²-Bench Telecom subset, and by 7.7% and 44.9% respectively on AppWorld, while maintaining accuracy. The code is publicly available on GitHub.", "body_md": "arXiv:2609.03236v1 Announce Type: new\nAbstract: Tool-using LLM agents spend wall-clock time not only on model inference but also in serial action--observation turns, where each tool call, environment transition, and observation can delay subsequent decisions. We introduce \\textbf{Speculative Macro Commit} (SMC), a runtime mechanism for a two-tier agent system: a large authoritative actor model produces the official trajectory, while a faster speculative drafter model continuously predicts and executes future action chains on an isolated environment snapshot. SMC mines recurring multi-action skeletons from training traces and stores them in a macro library used to match against action chains predicted by the drafter at runtime. When the actor's next tool call matches the first drafted action, SMC commits the remaining pre-executed draft steps, together with their observations, to the official trajectory. Using Qwen3.5-27B INT4 as the authoritative actor model and Qwen3.5-4B as the speculative drafter model, SMC matches the sequential agent's overall accuracy while reducing latency by 10.23\\% over the Speculative Actions (SA) baseline and 18.59\\% over sequential execution on the $\\tau^2$-Bench Telecom subset. On AppWorld, SMC reduces wall time by 7.7\\% over SA baseline and 44.9\\% over sequential execution, with a small reduction in task completion. Overall, SMC provides a practical way to reuse multi-step speculative execution and reduce agent latency beyond single-step speculative actions. Our code is publicly available \\href{https://github.com/zeyuliu1037/speculative-macro-commit}{\\textcolor{magenta}{here}}.", "url": "https://wpnews.pro/news/speculative-macro-commit-for-faster-tool-using-agents", "canonical_source": "https://arxiv.org/abs/2609.03236", "published_at": "2026-09-04 04:00:00+00:00", "updated_at": "2026-09-04 04:23:47.106387+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-agents", "ai-research"], "entities": ["arXiv", "Qwen3.5-27B INT4", "Qwen3.5-4B", "Speculative Macro Commit", "τ²-Bench", "AppWorld", "GitHub"], "alternates": {"html": "https://wpnews.pro/news/speculative-macro-commit-for-faster-tool-using-agents", "markdown": "https://wpnews.pro/news/speculative-macro-commit-for-faster-tool-using-agents.md", "text": "https://wpnews.pro/news/speculative-macro-commit-for-faster-tool-using-agents.txt", "jsonld": "https://wpnews.pro/news/speculative-macro-commit-for-faster-tool-using-agents.jsonld"}}