{"slug": "6-critics-grade-my-post-then-fix-the-skill-that-wrote-it", "title": "6 Critics Grade My Post, Then Fix the Skill That Wrote It", "summary": "Six adversarial critic subagents, dispatched in parallel by a Claude Code blog-post skill, returned two skill_adjustments that named the instruction file section responsible for a flaw, enabling a self-improving loop. The system, detailed by an unnamed author, pins each agent to a single failure mode via lane rules in agent files and requires exact quotes from indicted blocks, with the ai-detector lens filling a unique field so seven opinions merge by function. The approach solves the problem of generic reviewer notes that cannot edit instruction files because they never name the failed instruction.", "body_md": "July 23, 2026\n\nSix critics read my last draft in parallel. Two of their twenty findings were not about the draft at all: they came back as `skill_adjustments`\n\n, naming a section of the instruction file that wrote it. That is the trick behind a self-improving Claude Code skill. The loop writes the patch. I apply it by hand, and the hand-apply is load-bearing.\n\nRunning one general reviewer over the draft is where this usually dies. The notes come back generic, and a generic note cannot edit an instruction file, because it never names the instruction that failed.\n\nStage 5 of my blog-post skill dispatches six adversarial critic subagents in a single message, so they run in parallel and never read each other's output. The six are rules-judge, claim-auditor, expert-skeptic, hostile-reader, ai-detector, and contract-lens. Each one owns a single failure mode and is told in writing to stay out of the other lanes.\n\nThat lane rule lives in the agent files. The ai-detector's contract tells it the rules-judge already adjudicates regex-level hits, so it must not re-report an isolated one. Without that line, six reports collapse into the same report six times.\n\n```\n---\nname: blog-post-ai-detector\ndescription: Panel lens that reads a blog draft the way AI detectors and AI-literate readers do. Returns a detection-risk score.\ntools: Read, Grep, Glob\nmodel: opus\ncolor: red\n---\n\n<!-- ... -->\n\nDivision of labor: the rules-judge already adjudicates the script's regex\nhits (banned words, em dashes, \"However,\" openers). Do NOT re-report an\nisolated regex-level tell. You own what regexes cannot see: statistical\ntexture, structural templates, and pattern CLUSTERS.\n```\n\nModel is pinned per agent. The orchestrator can run on something fast and cheap while every lens runs on opus, so a quick driver never degrades the judgment.\n\n| Criterion | rules-judge | claim-auditor | expert-skeptic | hostile-reader | ai-detector | contract-lens |\n|---|---|---|---|---|---|---|\n| Adjudicates mechanical script hits in context | ● | ○ | ○ | ○ | ○ | ○ |\n| Traces every claim back to the research brief | ○ | ● | ◐ | ○ | ◐ | ◐ |\n| Steelmans the strongest technical objection | ○ | ○ | ● | ◐ | ○ | ◐ |\n| Drafts the top hostile comment verbatim | ○ | ○ | ○ | ● | ○ | ○ |\n| Scores AI detection risk | ◐ | ○ | ○ | ◐ | ● | ○ |\n| Grades the draft against this post's reader contract | ○ | ○ | ◐ | ◐ | ○ | ● |\n| Reviews the artifact plan before anything is built | ◐ | ○ | ○ | ○ | ◐ | ● |\n| Can file a BLOCKER | ● | ● | ○ | ○ | ○ | ● |\n\nAll six return the same JSON shape, with one field only the ai-detector fills. That is the only reason seven opinions can be merged by a function instead of by me reading seven reports at midnight.\n\nBoth live in the same report. A finding indicts a block of the draft. An adjustment indicts the instruction that produced it.\n\n```\n{\n  \"lens\": \"ai-detector\",\n  \"aiRisk\": { \"score\": 4, \"drivers\": [\"uniform paragraph shape across four sections\"] },\n  \"findings\": [\n    {\n      \"severity\": \"MAJOR\",\n      \"block\": 14,\n      \"evidence\": \"exact quote from the block being indicted\",\n      \"issue\": \"which signal fired and why it fires here\",\n      \"suggestion\": \"the rewritten human version of that sentence\"\n    }\n  ],\n  \"strengths\": [\n    { \"evidence\": \"exact quote\", \"why\": \"rhythm break, odd word, lived detail\" }\n  ],\n  \"wow\": { \"score\": 7, \"moments\": [\"the arbitration is a reducer, not a mood\"] },\n  \"skill_adjustments\": [\n    {\n      \"file\": \"agents/writer.md\",\n      \"section\": \"Inline Code Restraint\",\n      \"problem\": \"Writer re-backticked the same token on every mention, so one paragraph rendered as five code chips.\",\n      \"suggestion\": \"State the rule as first-mention-only and cap inline code at two spans per block.\"\n    }\n  ]\n}\n```\n\nEvery finding carries an exact quote from the block it indicts, and a lens that cannot produce the quote does not get to file the finding.\n\nScoring is deliberately hard to game. Every lens returns a wow score from 0 to 10 and the merge takes the median instead of the mean. The claim-auditor reads a walkthrough as if it were an argument, comes back at 2, and a mean would drag a perfectly good draft into the revise pile on one lens's category error. A median absorbs that, and it absorbs the opposite case too, where a single lens falls in love with a paragraph the other six walked past.\n\n`aiRisk`\n\nis the exception to the whole averaging scheme, because only the ai-detector returns it, so the merge copies that object across verbatim.\n\nThe last field is the one that changes anything. It names a file path and a section heading inside the instruction file that produced the draft.\n\nThe Inline Code Restraint entry above is one my own panel filed against `agents/writer.md`\n\n. Here is the paragraph that triggered it, and the same paragraph after the rule landed.\n\n```\nSet `chrome` to `terminal`, `editor`, `diff`, or `plain`, then make the\n`label` match it: `terminal` wants a shell context, `editor` wants a real\nfile path, and `diff` wants `before` or `after`.\nPick the `chrome` first, then make the label match it. A terminal frame\nwants a shell context, an editor frame wants a real file path, and a diff\nframe wants before or after.\n```\n\nNo framework is involved here. The [Claude Code skills docs](https://code.claude.com/docs/en/skills) describe a skill as a directory with `SKILL.md`\n\nas the entrypoint, YAML frontmatter plus markdown instructions. Editing it is a `Write`\n\ncall against a text file.\n\nHow one lens report reaches SKILL.md\n\n[Reflexion](https://arxiv.org/abs/2303.11366) is the closest published loop, and the difference is where the lesson gets stored. Its reflections live in an episodic memory buffer scoped to the current task run. skill_adjustments writes the durable instruction file that governs every future run.\n\nThe next run does not remember this post and does not need to. It starts from a repaired instruction file instead.\n\nCollect only faults and every pass makes the instruction file more restrictive. Nothing in the loop ever argues for keeping a move that worked. I have not watched a self-editing SKILL.md sand off its own edges yet, and I would rather not learn how many runs that takes.\n\n[Huang et al.](https://arxiv.org/abs/2310.01798) ran the experiment that settles this. Loops carrying an external correctness signal improve. Loops running on self-critique alone drift down, and the model becomes more likely to break a correct answer than to fix a wrong one.\n\nSo every lens returns a `strengths[]`\n\narray and a wow score next to its findings. Call that an oracle and I would be lying. Six prompts with disjoint mandates buy decorrelated error, and under Huang et al.'s own taxonomy a panel of same-family lenses is intrinsic self-correction wearing role prompts.\n\nThe genuinely external signals here are three: the non-Claude peer, the mechanical regex pass that runs before the panel, and me applying adjustments by hand. strengths[] is a ratchet brake on the instruction file getting monotonically more restrictive, which is a different job from telling me whether the draft is correct.\n\nA reinforcement adjustment reads like this.\n\n```\n{\n  \"file\": \"agents/writer.md\",\n  \"section\": \"Voice\",\n  \"problem\": \"none, this is a reinforcement entry\",\n  \"suggestion\": \"Keep naming the wrong approach before the right one. hostile-reader and ai-detector both cited it as a wow moment. Do not promote it into a per-section template, which is how it stops working.\"\n}\n```\n\nReinforcement entries are the ones I edit hardest before applying. A note that says keep doing X sits one careless merge away from a rule that says always do X, and always-do-X is how you get a template.\n\nNone of this matters if the skill never fires in the first place. Getting the description right so Claude actually invokes it is a different problem, and I covered it in [how to write skills that actually trigger](/blog/guides/create-claude-code-skills). The loop here presumes a skill that already runs.\n\nSix lenses plus an outside model from a different family run concurrently against the same draft. Seven reports come back and they disagree. A multi-model verdict has to be a function over the merged findings, because I am not arbitrating seven JSON files by hand.\n\nThe seventh reviewer runs outside the subagent pool entirely. It is a separate CLI call fired concurrently with the six dispatches, handed the rendered draft and the reader contract, and told to review as an outside editor.\n\n```\nbun ~/.claude/skills/rizz-partner/rizz.ts crew \"Peer-review blog draft\" \\\n  --task \"adversary: Read <run_dir>/draft.md and <run_dir>/reader-contract.json.\n          Review as an outside editor: the strongest objection to the thesis,\n          claims that smell wrong, sections that drag, the genuine wow moments,\n          a one-line verdict. Return JSON: {findings, strengths,\n          wow: {score, moments}, verdict}.\"\n```\n\nIts JSON is mapped to a lens report with `\"lens\": \"outside-peer\"`\n\nbefore the merge sees it. If the crew times out or returns something unparseable, the orchestrator logs an ADVISORY finding and the panel continues. The peer is a vote the pipeline can afford to lose.\n\nThe merge itself is an inline orchestrator step. SKILL.md spells it out in prose: dedupe findings citing the same block and issue, keep the highest severity, note the corroborating lenses. It is deterministic enough to write as a function, so here it is as one.\n\n```\ntype Severity = \"BLOCKER\" | \"MAJOR\" | \"ADVISORY\";\ntype Finding = { lens: string; severity: Severity; block: number; issue: string; evidence: string };\ntype Merged = Finding & { lenses: string[] };\n\nconst RANK: Record<Severity, number> = { ADVISORY: 0, MAJOR: 1, BLOCKER: 2 };\nconst key = (f: Finding) => `${f.block}::${f.issue.toLowerCase().slice(0, 60)}`;\n\nfunction mergeFindings(reports: { lens: string; findings: Finding[] }[]): Merged[] {\n  const byKey = new Map<string, Merged>();\n\n  for (const report of reports) {\n    for (const f of report.findings) {\n      // A different model family gets a vote. It does not get a veto.\n      const severity: Severity =\n        report.lens === \"outside-peer\" && f.severity === \"BLOCKER\" ? \"MAJOR\" : f.severity;\n\n      const k = key(f);\n      const prior = byKey.get(k);\n      if (!prior) {\n        byKey.set(k, { ...f, lens: report.lens, severity, lenses: [report.lens] });\n        continue;\n      }\n\n      prior.lenses.push(report.lens);\n      if (RANK[severity] > RANK[prior.severity]) prior.severity = severity;\n    }\n  }\n\n  return [...byKey.values()].sort((a, b) => RANK[b.severity] - RANK[a.severity]);\n}\n```\n\nLook at the cap. An outside-peer finding enters at MAJOR even when the peer files a BLOCKER.\n\nThat cap is a design response to a documented bias, not a benchmarked result. Nobody has evaluated severity-capping a cross-model reviewer. What is measured is judge reliability, and on expert-knowledge tasks [it drops below the inter-expert baseline](https://arxiv.org/pdf/2605.25273) that a panel of humans clears.\n\nPromotion back to BLOCKER runs through that same key. When a Claude lens independently files the same block and the same issue at BLOCKER, the merged entry already carries that severity, so corroboration needs no branch of its own.\n\nThe key under-merges, badly. Two lenses have to agree on the first 60 characters of a free-text issue string, which in practice they rarely do, so the promotion path fires less often than the code implies and I read near-duplicates by hand. Keying on the normalized evidence quote is the obvious fix and it is on my list.\n\n```\ntype Verdict = \"BLOCKED\" | \"REVISE\" | \"PUBLISH\";\n\nfunction verdictFor(findings: Merged[], aiRisk: { score: number }): Verdict {\n  if (findings.some((f) => f.severity === \"BLOCKER\")) return \"BLOCKED\";\n  if (findings.some((f) => f.severity === \"MAJOR\")) return \"REVISE\";\n  // aiRisk overrides a clean findings list.\n  if (aiRisk.score >= 5) return \"REVISE\";\n  return \"PUBLISH\";\n}\n```\n\nIs role-splitting actually better than one reviewer? [MARG](https://arxiv.org/abs/2401.04259) measured it on scientific paper review: role-decomposed agents cut generic comments from 60% to 29% against a single GPT-4 reviewer, and produced 3.7 good comments per paper against 1.7 for the best single-model baseline.\n\nRead the other side before you copy this. A 2026 paper finds [single-agent models beating multi-agent systems](https://arxiv.org/abs/2604.02460) on multi-hop reasoning under equal thinking-token budgets, and it finds the multi-agent edge shrinks as frontier models improve. Both results hold, and the honest reading is that the case for splitting review is stronger than the case for splitting reasoning.\n\nSeven reviewers is not free. The panel that reviewed this post burned 310,309 tokens across the six Claude lenses, and because they run in parallel the wall clock was the slowest one, a little over six minutes. I have never run the control that would justify that spend: one strong reviewer handed all six mandates at an equal token budget. That is the exact comparison the single-agent paper makes.\n\nAn agent rewriting its own instructions on an unbounded loop has a known ending. The literature name is [model collapse](https://www.nature.com/articles/s41586-024-07566-y): recursive training on model output produces irreversible defects where the tails of the original distribution disappear. The measurement is about training data. The shape of the risk is the same, and the name is worth borrowing.\n\nThe tallest bar on that chart is Self-Refine at 20 points, running on self-feedback with no external signal at all, which looks like it demolishes the argument until you notice it refines once and stops. The loops that gain do it in a single pass. The losses show up when the same loop runs round after round. That is what Huang et al. and the [FlipFlop experiment](https://arxiv.org/abs/2311.08596) measured. GPT-4 fell from 95.5% to 89.0% on GSM8K over two rounds.\n\nThese are the guardrails I actually run.\n\nThe docs are precise about one half of this. A plain personal or project skill under `~/.claude/skills/`\n\npicks up SKILL.md text edits live in the current session, no restart required.\n\nMy lenses mostly rewrite agent files inside a plugin, and a plugin's `agents/`\n\ndirectory sits outside live change detection, so those edits need a `/reload-plugins`\n\n. The version bump is my own addition. I bump it so the change lands in a commit before it lands in a run.\n\n```\n# The lenses rewrote agents/writer.md. A plugin's agents/ dir is outside\n# live change detection, so the running session will not see it yet.\njq '.version = \"0.5.1\"' .claude-plugin/plugin.json > plugin.tmp \\\n  && mv plugin.tmp .claude-plugin/plugin.json\n\ngit commit -am \"feat(blog): apply skill_adjustments from the 2026-07-23 run\"\n\n# /reload-plugins is what the docs require. The bump above is my own gate:\n# no adjustment reaches a run before it reaches a commit.\n```\n\nOne guardrail I do not have. Nothing in the loop has ever proposed a deletion, and reinforcement entries add text exactly the way fault entries do, so both channels are additive and the only subtractive force in this system is me with a text editor. A loop that edits its own instructions grows, which means the thing I claim corrects the process is also quietly inflating it.\n\nSix lenses is more than you need to start. Two will do, as long as each is graded on a single failure mode and both return the same shape.\n\nHalf the findings I dismiss on read. What earns the cost is the schema: seven independent readings against one shape make a bad instruction file visible, and a bad instruction file is invisible from inside the run it ruins.\n\nRun it once against a skill you already trust. Point it at a post you were happy with and see which of your own prompt lines come back named.", "url": "https://wpnews.pro/news/6-critics-grade-my-post-then-fix-the-skill-that-wrote-it", "canonical_source": "https://rizz.dev/blog/deep-dives/self-improving-claude-code-skill-6-critic-loop", "published_at": "2026-07-23 00:00:00+00:00", "updated_at": "2026-07-23 23:40:08.893419+00:00", "lang": "en", "topics": ["artificial-intelligence", "ai-agents", "ai-tools", "large-language-models"], "entities": ["Claude Code"], "alternates": {"html": "https://wpnews.pro/news/6-critics-grade-my-post-then-fix-the-skill-that-wrote-it", "markdown": "https://wpnews.pro/news/6-critics-grade-my-post-then-fix-the-skill-that-wrote-it.md", "text": "https://wpnews.pro/news/6-critics-grade-my-post-then-fix-the-skill-that-wrote-it.txt", "jsonld": "https://wpnews.pro/news/6-critics-grade-my-post-then-fix-the-skill-that-wrote-it.jsonld"}}