Jiabao Ji, Yujian Liu and Li An, three UC Santa Barbara researchers working as LinkedIn interns, trained AI agents to predict and pre-execute their own next tool calls, an attempt to remove a growing source of delay in systems that depend on search engines, databases and external APIs.
The researchers described the method in a July 28th arXiv paper, written with LinkedIn researchers Rohit Jain, Gungor Polatkan and Siyu Zhu and UCSB computer science professor Shiyu Chang. DAIR.AI highlighted the research on X on July 30th.
Ji is a fourth-year UCSB Ph.D. student advised by Chang. His previous research internships include Adobe Research, Meta's GenAI Llama group and Apple AI/ML, followed by work with LinkedIn's CoreAI group in 2026. Liu, also a fourth-year Ph.D. student advised by Chang, previously worked at Adobe Research, the MIT-IBM Watson AI Lab and AMD GenAI. Their new paper moves beyond training agents to choose tools correctly and focuses on when those calls can be started.
One model, two jobs
Tool-using agents alternate between generating text and waiting for outside systems. A search request, database lookup or sub-agent can take longer than the model's own token generation, leaving expensive inference hardware idle and making an otherwise capable agent feel slow.
Speculative tool execution tries to fill that gap. While the agent continues reasoning, a speculator predicts the next structured call, including its tool name and arguments, and starts the request early. If the prediction exactly matches the call the agent eventually produces, the system reuses the result. A mismatch means the speculative work must be discarded.
Previous approaches have commonly assigned that prediction to a smaller draft model or retrieved a likely action from cached traces. The UCSB and LinkedIn researchers argue that those systems create a "speculator-agent gap": the draft model approximates what the agent might do without precisely matching the deployed agent's policy. The extra model also requires its own weights and KV cache during serving.
Their self-speculating design gives both jobs to one language model. In agent mode, the model reasons through the task and calls tools normally. In speculator mode, it receives a partial trajectory plus a short suffix asking it to predict the next structured call. Both modes share model parameters and can reuse the same prefix KV cache.
The researchers trained the two modes through alternating reinforcement-learning updates. Each batch of agent rollouts produces fresh examples of the calls the current model actually chose. Those calls then become targets for its speculation training. The final training schedule used four agent updates followed by eight speculator updates, with the optimizer state reset whenever training switched modes.
That separation was consequential. A one-to-one update schedule produced an average Hit@1 score of 31.8 and task success of 10.3 across the three search benchmarks used in the ablation. The four-to-eight schedule reached 55.2 Hit@1 and 26.1 task success under the same iteration budget.
The benchmark gains
The researchers tested the method with the 4 billion-parameter Qwen3 and Qwen3.5 models. Evaluation covered HotpotQA, MuSiQue and BrowseComp-Plus for search, along with the airline and retail domains of tau-bench for conversational API use.
For Qwen3-4B, average next-call Hit@1 rose from 44.1 after supervised fine-tuning to 61.2 after the joint reinforcement-learning stage. Qwen3.5-4B improved from 48.9 to 66.3. A Hit@1 success required an exact match on both the tool name and the complete argument dictionary. The reported downstream task-success averages also held steady or increased slightly. Qwen3-4B moved from 26.6 after supervised fine-tuning to 27.7 after reinforcement learning, while Qwen3.5-4B moved from 49.2 to 50.6. Those averages combine tasks with substantially different baseline difficulty, so they establish that the added speculation objective did not erase the agent's existing capabilities in the tested settings.
An earlier off-the-shelf comparison showed why the researchers rejected a separate draft model. On MuSiQue, Qwen3-4B predicting its own next call reached 25.3 Hit@1 while using 8.70 GB of GPU memory and 8.3 seconds of speculation wall-clock time. A Qwen3-1.7B external speculator reached 14.7 Hit@1 while the combined setup consumed 12.76 GB and 33.1 seconds. The tests ran on a single Nvidia H100 and used explicit switching between models, so those measurements describe the paper's serving setup rather than a universal production cost.
Where the claim stops
The results establish higher prediction accuracy and lower serving overhead for the self-speculating configuration. The paper does not report an end-to-end production deployment showing how much user-visible latency the trained model removes when real tools run concurrently.
Exact matches also occurred in roughly six to seven of every ten predictions after training. The remaining speculative calls could not be reused, leaving operators to weigh wasted API requests and compute against the latency saved by correct guesses.
Safety narrows the immediate use cases further. A mistaken speculative web search can be discarded. A mistaken order, database update or outbound message can change external state before the agent has committed to the action. The authors limit their current case to read-only operations and suggest dry-run modes, rollback mechanisms or human confirmation for state-changing tools.
The experiments used 4B-scale models and two task families. Code execution, browser control, long-running workflows, multi-agent systems and larger frontier models remain outside the evaluation. Even within that boundary, the work identifies a practical design choice for agent builders: the deployed policy may be a better predictor of its next action than a cheaper model trained to imitate it.