{"slug": "running-the-blog-s-content-autopilot-on-a-claude-subscription-not-api-credit", "title": "Running the blog's content autopilot on a Claude subscription, not API credit", "summary": "Developer Russell Jones has documented how his blog's Content Autopilot workflow was hardened after going live, including switching from per-token API billing to a Claude subscription via the CLAUDE_CODE_OAUTH_TOKEN environment variable. The workflow now includes failure alerts that create GitHub issues when authentication fails, and a self-refine step that critiques and revises drafts before publishing to catch low-quality output.", "body_md": "Ahnii!\n\n[https://jonesrussell.github.io/blog/content-autopilot/](https://jonesrussell.github.io/blog/content-autopilot/) covers how this blog's [Content Autopilot](https://github.com/jonesrussell/blog/blob/main/.github/workflows/content-autopilot.yml) workflow mines, curates, and publishes without a human reading the draft first. Three things broke or needed hardening after that pipeline went live: how it pays for Claude, why the auth silently died, and the two gates bolted on afterward to catch a bad draft before it ships.\n\nThe produce step originally authenticated with `ANTHROPIC_API_KEY`\n\n, a pay-as-you-go API account billed per token. That account ran dry, and the daily cron started failing on the very first Claude call. Swapping to `CLAUDE_CODE_OAUTH_TOKEN`\n\nfixed it in one line: this env var runs the headless CLI against a Claude subscription instead.\n\n```\nenv:\n  # Subscription auth (no per-token API billing). See CLAUDE_CODE_OAUTH_TOKEN secret.\n  CLAUDE_CODE_OAUTH_TOKEN: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }}\n  GH_TOKEN: ${{ secrets.CROSS_REPO_TOKEN }}\n```\n\n`claude setup-token`\n\nmints this token locally. It's a short-lived credential meant for one CLI session, not a service account, and that mismatch is exactly what caused the next failure.\n\nSubscription tokens rotate. This one got revoked, and the workflow had no step watching for a `failure()`\n\noutcome. Four daily runs failed in a row before anyone noticed, because a cron job with no output channel just... stops. Nobody was checking the Actions tab every morning.\n\nThe fix wasn't a longer-lived token. It was making failure loud:\n\n```\n- name: Alert on failure\n  if: failure()\n  env:\n    GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}\n    REPO: ${{ github.repository }}\n    RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}\n  run: |\n    gh label create autopilot-failure --repo \"$REPO\" --color d73a4a \\\n      --description \"Content Autopilot run failed\" 2>/dev/null || true\n    body=$(printf 'Content Autopilot run failed: %s\\n\\nMost common cause: CLAUDE_CODE_OAUTH_TOKEN expired or was revoked (subscription tokens rotate). Re-sync the secret from a fresh token, or run `claude setup-token` for a long-lived one.\\n\\n@jonesrussell' \"$RUN_URL\")\n    existing=$(gh issue list --repo \"$REPO\" --label autopilot-failure --state open --json number --jq '.[0].number // empty')\n    if [ -n \"$existing\" ]; then\n      gh issue comment \"$existing\" --repo \"$REPO\" --body \"$body\"\n    else\n      gh issue create --repo \"$REPO\" --title \"Content Autopilot is failing\" \\\n        --label autopilot-failure --body \"$body\"\n    fi\n```\n\nThis runs on any step failure in the job, not just the auth step. It opens one issue, reuses it on repeat failures instead of spamming a new one per day, and GitHub's default notification settings turn that issue into an email. The `permissions`\n\nblock needed `issues: write`\n\nadded alongside the existing `contents: write`\n\nand `id-token: write`\n\nfor this to work.\n\nAuth failures are loud and easy to fix. A boring, over-hedged, cliche-riddled post is a quieter failure, and nothing upstream of publish was checking for it. The next addition borrows the Self-Refine technique from the AI-writing research: have the model critique its own output once, then revise, before anything ships.\n\n```\nCritique and then revise the blog post at <path> and its social copy.\nRubric: (1) voice matches docs/blog-style.md and the essay reference;\n(2) every claim is backed by a concrete fact, number, or reference;\n(3) sentence lengths vary (no runs of same-length sentences);\n(4) zero phrases from the site's banned AI-cliche list;\n(5) intro scope line is not the phrase 'This post covers'.\n```\n\nThe step is `continue-on-error: true`\n\n. If the critique call itself fails or times out, the workflow falls back to publishing the un-refined first draft rather than blocking the whole run over a quality pass.\n\nA revise pass helps, but it's still the same model marking its own homework. The step after it is deliberately dumb: `scripts/slop-check.mjs`\n\nruns no API call at all. It strips frontmatter, code fences, and markdown syntax, splits what's left into sentences, and scores three signals.\n\n``` js\nconst mean = totalWords / lens.length;\nconst variance = lens.reduce((a, n) => a + (n - mean) ** 2, 0) / lens.length;\nconst stdev = Math.sqrt(variance);\nconst burstiness = stdev / mean; // human ~0.6-1.2, AI slop < 0.4\n```\n\nHuman writing varies sentence length a lot; AI writing tends to cluster around one length. The script also counts em dashes per thousand words (over 20 reads as machine-written) and matches a growing list of stock corporate-blog phrases pulled straight from the site's own style guide. It fails the draft on a hard burstiness floor of 0.32. Three or more banned-phrase hits alone are enough too, and so is any two of the three signals firing together. A failed check holds the draft: nothing gets committed, and the source issue stays open for the next day's run to try again.\n\nBuild order matters here. The slop gate runs before the Hugo build gate, so a flagged draft never even reaches `hugo --gc --minify`\n\n, let alone `git push`\n\n.\n\nTwo posts have shipped through this pipeline since it went live, tracked by issue number in `data/autopilot-ledger.json`\n\n. None of the hardening above came from planning ahead. Each piece exists because a specific run failed in a specific way: out-of-balance API credit, then a revoked token nobody caught, then no evidence-based reason to trust an unreviewed draft's prose. Unsupervised doesn't mean untested. It means every failure mode has to turn into a gate before the same thing is allowed to happen twice.\n\nBaamaapii", "url": "https://wpnews.pro/news/running-the-blog-s-content-autopilot-on-a-claude-subscription-not-api-credit", "canonical_source": "https://dev.to/jonesrussell/running-the-blogs-content-autopilot-on-a-claude-subscription-not-api-credit-4g2p", "published_at": "2026-08-21 23:52:30+00:00", "updated_at": "2026-08-22 00:14:03.573608+00:00", "lang": "en", "topics": ["developer-tools", "ai-tools", "mlops", "generative-ai"], "entities": ["Russell Jones", "Claude", "GitHub", "Content Autopilot"], "alternates": {"html": "https://wpnews.pro/news/running-the-blog-s-content-autopilot-on-a-claude-subscription-not-api-credit", "markdown": "https://wpnews.pro/news/running-the-blog-s-content-autopilot-on-a-claude-subscription-not-api-credit.md", "text": "https://wpnews.pro/news/running-the-blog-s-content-autopilot-on-a-claude-subscription-not-api-credit.txt", "jsonld": "https://wpnews.pro/news/running-the-blog-s-content-autopilot-on-a-claude-subscription-not-api-credit.jsonld"}}