cd /news/artificial-intelligence/skills-x2-108-runs-optimising-effica… · home topics artificial-intelligence article
[ARTICLE · art-99534] src=news.ycombinator.com ↗ pub= topic=artificial-intelligence verified=true sentiment=· neutral

Skills x2, 108 Runs: Optimising Efficacy and Token Efficiency

A developer's benchmark of coding agents on 108 bugs found that models memorized answers, raising questions about evaluation efficacy. The Signal Bench, created by darvh, showed high scores but flagged memorization, prompting a call for community feedback on the benchmark's methodology.

read1 min views6 publishedAug 17, 2026

| |||||||||||| 1 point by | Ponytail vs Signal Bench has the runs albeit not in an organised manner. Models showed memorization of the answers, which is kinda expected these days. More than the skills, it was fun getting the benchmark running :). Let me know if the preamble on how the bench was conducted or constructed is flawed. Blog Post: darvh.com/posts/when-coding-agents-raced-through-108-bugs/ Bench: github.com/darvh/bench Signal: github.com/darvh/signal | ||||||||||| |

── more in #artificial-intelligence 4 stories · sorted by recency
── more on @signal bench 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
Live at https://your-agent.zahid.host
Get free account → Pricing
from €0/mo · no card required
LIVE [news/skills-x2-108-runs-o…] indexed:0 read:1min 2026-08-17 ·