cd /news/ai-agents/compskill-compiling-execution-experi… · home › topics › ai-agents › article
[ARTICLE · art-163638] src=machinebrief.com ↗ pub= topic=ai-agents verified=true sentiment=↑ positive

CompSkill: Compiling Execution Experience for Skill Retrieval

CompSkill, a skill-retrieval method detailed in arXiv paper 2611.01115v1, raises SkillRet NDCG@10 from 64.0 to 70.9 at one eighth of the key dimension by indexing skills with learned 512-d keys and compiling matched same-task comparisons into skill offsets and capped key residuals. With a single document, CompSkill completes 256 and 260 of 332 ToolQA tasks using GPT-5.6-luna and DeepSeek-V4.1-Flash respectively, beating every baseline, and on 332 newly generated ToolQA tasks it completes 256 with GPT-5.6-luna, more than BM25, SkillRet-8B, Tool2Vec and RankGPT; across all 844 ToolQA tasks it completes 24 more than the strongest baseline. The authors report that pairing the right guide with a same-database sibling rather than another database's guide costs 8 of 96 tasks, and released code, models and support data on GitHub.

by read1 min views7 publishedNov 4, 2026

arXiv:2611.01115v1 Announce Type: new Abstract: LLM agents increasingly rely on skill documents, yet most retrievers rank them by description alone and ignore what earlier executions revealed, even though a relevant skill can hurt execution. We present CompSkill, which indexes skills with learned 512-d keys and compiles matched same-task comparisons into skill offsets and capped key residuals (KeyOpt), so experience changes which document the executor receives, never its text. Under a logistic outcome model with unknown task difficulty, matched comparisons cancel that difficulty and only discordant ones carry information; on support executions, predictions improve with each discordant task. An experience hypergraph keeps each comparison with its executor and document versions, recalls missed skills and grounds a cited review. Compact keys raise SkillRet NDCG@10 from 64.0 to 70.9 at one eighth of the dimension. With one document, CompSkill completes 256 and 260 of 332 ToolQA tasks with GPT-5.6-luna and DeepSeek-V4.1-Flash, respectively, more than every baseline, and stays on par with the strongest baselines on AppWorld and ALFWorld. On 332 newly generated ToolQA tasks, it completes 256 with GPT-5.6-luna, significantly more than BM25, SkillRet-8B, Tool2Vec and RankGPT; over all 844 ToolQA tasks, it completes 24 more than the strongest baseline. Selection matters beyond relevance: pairing the right guide with a same-database sibling instead of another database's guide costs 8 of 96 tasks. Code, models and support data: https://github.com/satoshinji2992/CompSkill

── more in #ai-agents 4 stories · sorted by recency
── more on @compskill 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
→ Live at https://your-agent.zahid.host ✓
Get free account → Pricing
from €0/mo · no card required
LIVE [news/compskill-compiling-…] indexed:0 read:1min 2026-11-04 · —