{"slug": "uncertainty-quantification-for-llm-function-calling", "title": "Uncertainty Quantification for LLM Function-Calling", "summary": "Researchers at Apple and other institutions published the first evaluation of uncertainty quantification methods for LLM function-calling, finding that multi-sample methods like Semantic Entropy offer no clear advantage over simpler single-sample approaches. The study proposes improvements leveraging function-calling output structure, such as clustering based on abstract syntax tree parsing and selecting semantically meaningful tokens for logit-based scores.", "body_md": "[content type paper](/research/)published July 2026\n\nUncertainty Quantification for LLM Function-Calling\n\nAuthorsZihuiwen Ye†*, Lukas Aichberger†*, Michael Kirchhof, Sinead Williamson, Luca Zappella, Yarin Gal†, Arno Blaas‡, Adam Goliński‡\n\nUncertainty Quantification for LLM Function-Calling\n\nAuthorsZihuiwen Ye†*, Lukas Aichberger†*, Michael Kirchhof, Sinead Williamson, Luca Zappella, Yarin Gal†, Arno Blaas‡, Adam Goliński‡\n\nLarge Language Models (LLMs) are increasingly deployed to autonomously solve real-world tasks. A key ingredient for this is the LLM Function-Calling paradigm, a widely used approach for equipping LLMs with tool-use capabilities. However, an LLM calling functions incorrectly can have severe implications, especially when their effects are irreversible, e.g., transferring money or deleting data. Hence, it is of paramount importance to consider the LLM’s confidence that a function call solves the task correctly prior to executing it. Uncertainty Quantification (UQ) methods can be used to quantify this confidence and prevent potentially incorrect function calls. In this work, we present what is, to our knowledge, the first evaluation of UQ methods for LLM Function-Calling (FC). While multi-sample UQ methods, such as Semantic Entropy, show strong performance for natural language Q&A tasks, we find that in the FC setting, it offers no clear advantage over simple single-sample UQ methods. Additionally, we find that the particularities of FC outputs can be leveraged to improve the performance of existing UQ methods in this setting. Specifically, multi-sample UQ methods benefit from clustering FC outputs based on their abstract syntax tree parsing, while single-sample UQ methods can be improved by selecting only semantically meaningful tokens when calculating logit-based uncertainty scores.\n\nRevisiting Uncertainty Quantification Evaluation in Language Models: Spurious Interactions with Response Length Bias Results\n\nJune 27, 2025[research area Methods and Algorithms](/research/?domain=Methods%20and%20Algorithms), [research area Speech and Natural Language Processing](/research/?domain=Speech%20and%20Natural%20Language%20Processing)[conference ACL](/research/?event=ACL)\n\nUncertainty Quantification (UQ) in Language Models (LMs) is key to improving their safety and reliability. Evaluations often use metrics like AUROC to assess how well UQ methods (e.g., negative sequence probabilities) correlate with task correctness functions (e.g., ROUGE-L). We show that mutual biases—when both UQ methods and correctness functions are biased by the same factors—systematically distort evaluation. First, we formally prove that…\n\nEfficient and Effective Uncertainty Quantification in LLMs\n\nNovember 21, 2024[research area Speech and Natural Language Processing](/research/?domain=Speech%20and%20Natural%20Language%20Processing)[Workshop at NeurIPS](/research/?event=NeurIPS%20Workshop)\n\nThis paper was accepted at the Safe Generative AI Workshop (SGAIW) 2024 at NeurIPS 2024.\n\nUncertainty quantification (UQ) is crucial for ensuring the safe deployment of large language model, particularly in high-stakes applications where hallucinations can be harmful. However, existing UQ methods often demand substantial computational resources, e.g., multi-sample methods such as Semantic Entropy (Kuhn et al., 2023) usually require 5-10 inference…", "url": "https://wpnews.pro/news/uncertainty-quantification-for-llm-function-calling", "canonical_source": "https://machinelearning.apple.com/research/uncertainty-quantification-function-calling", "published_at": "2026-07-15 00:00:00+00:00", "updated_at": "2026-07-15 21:17:48.022780+00:00", "lang": "en", "topics": ["large-language-models", "ai-safety", "ai-research"], "entities": ["Apple", "Zihuiwen Ye", "Lukas Aichberger", "Michael Kirchhof", "Sinead Williamson", "Luca Zappella", "Yarin Gal", "Arno Blaas"], "alternates": {"html": "https://wpnews.pro/news/uncertainty-quantification-for-llm-function-calling", "markdown": "https://wpnews.pro/news/uncertainty-quantification-for-llm-function-calling.md", "text": "https://wpnews.pro/news/uncertainty-quantification-for-llm-function-calling.txt", "jsonld": "https://wpnews.pro/news/uncertainty-quantification-for-llm-function-calling.jsonld"}}