{"slug": "show-hn-skillwire-a-searchable-catalog-for-coding-agent-skills", "title": "Show HN: SkillWire, a searchable catalog for coding-agent skills", "summary": "SkillWire launched as a searchable catalog of coding-agent skills that lets developers pin a single skill, inspect its source and download its ZIP, then run a local A/B evaluation using their existing Codex or Claude Code subscription rather than a hosted service or extra API key. In a first trial of ten matched pairs with a pinned public TDD skill folder, recorded task passes were 8 of 10 with the skill installed and 7 of 10 without, though the skill body appeared in only one treatment tool trace and nine reads were unobserved, leaving internal use unknown. SkillWire notes that a frozen reference accepted ASCII DEL where the task said to reject it, so that first family of results is disputed, and that one pair cannot establish general benefit.", "body_md": "**1 · Pick and pin**\n\nChoose exactly one skill in Search, inspect its source, then download its ZIP. Keep that ZIP for this trial.\n\n| Stack | Popularity | Name | Problem solved | Category | Repository | Rating | Eval | \n|---|---|---|---|---|---|---|---|\n\n`SKILL-SOURCES.json`, every warning line, the pinned repository licence and any skill-specific terms.`.agents/skills/` inside your project. Copy each reviewed skill folder there, keeping its `SKILL.md` and supporting files together. Use the skill name as its folder name. Do not replace an existing skill without comparing it.\nDownloaded scripts may run with your agent’s permissions if you later use them. Start in a disposable project with no private files or credentials. Other harnesses use different install locations; follow their official instructions.\n\nUse one pinned skill and your own disposable project. SkillWire freezes its exact files, your task, grader, CLI settings and arm order before the two local runs. The same starting files go to both arms. Review the artifacts blind before you reveal which arm had the skill.\n\nRun `python3 lime_agent_eval.py doctor --project . > doctor.json` from the downloaded trial kit, then open that file here. The CLI reports skill names and SKILL.md hashes without a model call. This browser cannot scan your files; the JSON stays in this tab.\n\nUse your task, starting files, a known good result, a deliberately bad result and an independent grader. The local preflight must pass good and reject bad.\n\nUse the same downloaded ZIP with `--stack skillwire-skill-stack.zip`. Seal the task, grader, model, arm order and same starting files; run the two arms on your subscription.\n\nOpen the saved A/B outputs without arm names, grades or commands. Save your choice, including tie or cannot judge, before revealing the conditions.\n\nSave a verdict in the blind reader. Then choose its `blind-review.json`, your saved `human-review-verdict.json`, and the withheld `reveal-key.json`. The key must match the blind bundle and the run receipts. This is a local consistency check, not proof of who judged.\n\nBlind verdict and reveal key required before labelled evidence.\n\nChoose the complete run folder for raw CLI trace evidence, or select its six JSON receipts: `plan.json`, `protocol.json`, `protocol-freeze.json`, `SKILL-SOURCES.json`, `artifact-results.json`, and `model-runs/records.json`. Missing traces stay unverified. Files stay in this browser.\n\nNo run loaded.\n\nA valid seal and completed CLI trace document what ran; they do not prove the model used the skill internally. One pair cannot establish general benefit. Missing arms or grader errors remain visible and are never scored as task failures. A historical run for another skill is labelled as a mismatch with your current selection.\n\nUse your existing Codex or Claude Code subscription and a selected installed skill or public stack. Freeze the task and independent outcome checks, then compare the same starting files with and without those instructions. No extra API key or hosted evaluation service.\n\n[Set up a local evaluation](eval-guide.html) [Download CLI tools and example](local-eval-tools.zip)\n\nLocal detection does not start calls. Explicit trials use your chosen provider and subscription allowance. Unknown/API-key authentication is refused. Local trial setup freezes settings and disables unrelated customizations to keep the comparison controlled.\n\nAutomated checks can miss usefulness, clarity and mistakes that matter to you. Make a blind A/B choice from the saved files before seeing which condition produced them. Ties and “cannot judge” are valid outcomes.\n\nTen matched pairs completed with a pinned public TDD skill folder. Recorded task passes were 8 of 10 with it installed and 7 of 10 without. The skill body was seen in one treatment tool trace; nine reads were unobserved, and internal use is unknown. A frozen reference accepted ASCII DEL where the task said to reject it, so the first family is disputed. A new ten-case contract is sealed for review with no model runs. These counts are not a skill benefit claim. Blind human choices have not been collected.\n\nThe August SkillsBench entry reported that the skill body opened on only 3 of 8 panel tasks. Its treated versus baseline solve rates showed no detectable lift, and the reported effect was within its A/A noise. Raw rollouts were later deleted, so those figures cannot be rechecked from the repository. The current SkillWire pilot is a separate native Codex task with an 8/8 tie. Do not combine their counts or infer a general result.\n\nRead in this browser only; no upload, publication or catalog score. Imported reports are not independently authenticated.\n\nWhich skill earned a place in your stack? Vote for one you’ve tried.\n\nOne vote each day, resetting at midnight UTC. No login. Browser and shared-network limits make repeat voting harder, not impossible. Votes are preferences, not verified installs or test results.\n\nLoading community votes…\n\nA useful skill depends on the job. These picks come from source review, concrete workflows and available evaluation evidence. They are not ordered by stars or paid placement.\n\nAuthor examples, our own trials and popularity are different kinds of evidence. Read the limits, then try a matched task.", "url": "https://wpnews.pro/news/show-hn-skillwire-a-searchable-catalog-for-coding-agent-skills", "canonical_source": "https://skillwire.net", "published_at": "2026-10-30 22:18:13+00:00", "updated_at": "2026-10-30 22:46:54.941131+00:00", "lang": "en", "topics": ["ai-agents", "ai-tools", "developer-tools", "ai-research"], "entities": ["SkillWire", "Codex", "Claude Code", "lime_agent_eval.py"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/show-hn-skillwire-a-searchable-catalog-for-coding-agent-skills", "markdown": "https://wpnews.pro/news/show-hn-skillwire-a-searchable-catalog-for-coding-agent-skills.md", "text": "https://wpnews.pro/news/show-hn-skillwire-a-searchable-catalog-for-coding-agent-skills.txt", "jsonld": "https://wpnews.pro/news/show-hn-skillwire-a-searchable-catalog-for-coding-agent-skills.jsonld"}}