{"slug": "carbon-aware-routing-for-function-calling-in-edge-cloud-llm-systems", "title": "Carbon-Aware Routing for Function Calling in Edge-Cloud LLM Systems", "summary": "A carbon-aware routing framework that distributes function-calling queries across a three-tier edge-cloud architecture reduces operational carbon emissions by 4x on average while matching cloud-level accuracy, according to an arXiv paper (arXiv:2609.13559v1). The framework uses a lightweight k-NN predictor in a unified semantic-lexical embedding space to estimate query-specific accuracy, delay, and power consumption on each edge tier, then combines those predictions with real-time grid carbon intensity to route each query to the lowest-emission tier that can execute it successfully. The authors evaluated the approach on state-of-the-art function-calling benchmarks and LLM families.", "body_md": "arXiv:2609.13559v1 Announce Type: new \nAbstract: Large Language Models (LLMs) with function-calling capabilities are becoming critical for modern agentic AI systems. Nevertheless, current deployments typically route inferences to powerful cloud-based models, incurring significant energy use and carbon emissions. We address this sustainability challenge with a carbon-aware routing framework that distributes function-calling queries across a three-tier edge-cloud architecture, combining edge and cloud LLMs on heterogeneous hardware. At its core, a lightweight k-NN predictor operating in a unified semantic-lexical embedding space estimates query-specific accuracy, delay, and power consumption on each edge tier. These predictions are then combined with real-time grid carbon intensity to route every query to the lowest-emission tier capable of executing it successfully. Evaluated on state-of-the-art function-calling benchmarks and LLM families, our framework matches cloud-level accuracy while reducing operational carbon emissions by $4\\times$ on average.", "url": "https://wpnews.pro/news/carbon-aware-routing-for-function-calling-in-edge-cloud-llm-systems", "canonical_source": "https://arxiv.org/abs/2609.13559", "published_at": "2026-09-15 04:00:00+00:00", "updated_at": "2026-09-15 04:35:45.341817+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-infrastructure", "ai-research", "ai-agents"], "entities": ["arXiv", "k-NN predictor"], "alternates": {"html": "https://wpnews.pro/news/carbon-aware-routing-for-function-calling-in-edge-cloud-llm-systems", "markdown": "https://wpnews.pro/news/carbon-aware-routing-for-function-calling-in-edge-cloud-llm-systems.md", "text": "https://wpnews.pro/news/carbon-aware-routing-for-function-calling-in-edge-cloud-llm-systems.txt", "jsonld": "https://wpnews.pro/news/carbon-aware-routing-for-function-calling-in-edge-cloud-llm-systems.jsonld"}}