DelegateEval: Evaluating Platforms for Personal External Service Agents

A November 2026 arXiv paper (2611.07508v1) proposes DelegateEval, a platform-level evaluation framework for "personal external service agents" — AI agents authorized and continuously managed by an individual to accept client requests, deliver work, and provide ongoing support using that person's knowledge and service boundaries. DelegateEval covers eight dimensions: ease of creation, service discovery and matching, delivery quality, expertise inheritance, service continuity, learning and improvement, reliability and governance, and efficiency and value. The authors present only the framework and an experimental design, stating that complete datasets, executable environments, a unified scoring protocol, and cross-model results remain to be developed and empirically validated.

arXiv:2611.07508v1 Announce Type: new Abstract: AI agents that work for an individual over time may also help that individual serve external clients. We define a personal external service agent as an agent authorized and continuously managed by a specific individual to accept client requests, deliver work, and provide ongoing support using that person's knowledge, experience, professional methods, and service boundaries. Such agents could allow individuals to serve more clients with less direct involvement and generate economic value at scale. We propose DelegateEval, a platform-level evaluation framework for agent systems that may combine large language models, reusable skills, protocols such as MCP, and executable tool environments. It covers eight dimensions: ease of creation, service discovery and matching, delivery quality, expertise inheritance, service continuity, learning and improvement, reliability and governance, and efficiency and value. These dimensions assess whether a platform enables people with different backgrounds to turn their expertise into ongoing services that deliver verifiable work, develop capabilities through experience, and remain economically feasible. We present the framework and an experimental design. Complete datasets, executable environments, a unified scoring protocol, and cross-model results remain to be developed and empirically validated.