cd /news/ai-tools/request-to-register-dedeprogames-pho… · home › topics › ai-tools › article
[ARTICLE · art-171973] src=discuss.huggingface.co ↗ pub= topic=ai-tools verified=true sentiment=· neutral

Request to register DedeProGames/phone-tool-bench as an official benchmark on the Hub

A developer posting as DedeProGames has asked Hugging Face to register DedeProGames/phone-tool-bench as an official benchmark on the Hub, submitting a held-out phone tool-calling benchmark of 520 items spanning English, Portuguese, Spanish and German over the full 108-tool phone catalog from LiquidAI/phone-assistant-tools. Scoring is deterministic exact match of the predicted tool-call list with no LLM judge, and the headline metric is macro exact match across seven item categories (single, multi, reasoning, lookalike, partial, no_tool, clarify); the eval.yaml at the repository root was validated at push time with tasks phone_tool_bench, config default and split test. The benchmark ships its own phonebench harness with unit tests plus tools.json and scoring.json, and a first result for klein-ml/KV1-1.2B at 12.5% macro exact match is documented with full traces in DedeProGames/kv1-1.2B-phone-tool-bench-results.

recommendation

Score tool-calling agents with deterministic exact match on the full predicted call list and report macro averages across categories like single, multi, lookalike, partial, no_tool and clarify, as PhoneToolBench does, rather than relying on an LLM judge.

more in playbook: Evals and testing for LLM apps
read1 min views4 publishedNov 18, 2026

Hello HF team,

I would like to request the registration of my dataset as a benchmark on the Hugging Face Hub:

Dataset: DedeProGames/phone-tool-bench ← (link 1: DedeProGames/phone-tool-bench · Datasets at Hugging Face) PhoneToolBench is a held-out benchmark for phone tool calling: 520 items across

English, Portuguese, Spanish and German, over the full 108-tool phone catalog from

LiquidAI/phone-assistant-tools. Scoring is deterministic exact match of the predicted

tool-call list (no LLM judge); the headline metric is macro exact match over the

seven item categories (single, multi, reasoning, lookalike, partial, no_tool, clarify).

The eval.yaml file is already at the repository root and was validated at push time

(tasks: phone_tool_bench, config: default, split: test). The benchmark ships its own harness (phonebench) with unit tests, plus tools.json and scoring.json for

reproduction. A first result (klein-ml/KV1-1.2B, macro 12.5%) is documented with full

traces here: ← (link 2: DedeProGames/kv1-1.2B-phone-tool-bench-results · Datasets at Hugging Face) Could you please add it to the benchmark allow-list? Happy to adjust the eval.yaml

if any schema changes are needed. Thank you!

── more in #ai-tools 4 stories · sorted by recency
── more on @hugging face 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
→ Live at https://your-agent.zahid.host ✓
Get free account → Pricing
from €0/mo · no card required
LIVE [news/request-to-register-…] indexed:0 read:1min 2026-11-18 · —