Running the blog's content autopilot on a Claude subscription, not API credit

Developer Russell Jones has documented how his blog's Content Autopilot workflow was hardened after going live, including switching from per-token API billing to a Claude subscription via the CLAUDE_CODE_OAUTH_TOKEN environment variable. The workflow now includes failure alerts that create GitHub issues when authentication fails, and a self-refine step that critiques and revises drafts before publishing to catch low-quality output.

Ahnii https://jonesrussell.github.io/blog/content-autopilot/ https://jonesrussell.github.io/blog/content-autopilot/ covers how this blog's Content Autopilot https://github.com/jonesrussell/blog/blob/main/.github/workflows/content-autopilot.yml workflow mines, curates, and publishes without a human reading the draft first. Three things broke or needed hardening after that pipeline went live: how it pays for Claude, why the auth silently died, and the two gates bolted on afterward to catch a bad draft before it ships. The produce step originally authenticated with ANTHROPIC API KEY , a pay-as-you-go API account billed per token. That account ran dry, and the daily cron started failing on the very first Claude call. Swapping to CLAUDE CODE OAUTH TOKEN fixed it in one line: this env var runs the headless CLI against a Claude subscription instead. env: Subscription auth no per-token API billing . See CLAUDE CODE OAUTH TOKEN secret. CLAUDE CODE OAUTH TOKEN: ${{ secrets.CLAUDE CODE OAUTH TOKEN }} GH TOKEN: ${{ secrets.CROSS REPO TOKEN }} claude setup-token mints this token locally. It's a short-lived credential meant for one CLI session, not a service account, and that mismatch is exactly what caused the next failure. Subscription tokens rotate. This one got revoked, and the workflow had no step watching for a failure outcome. Four daily runs failed in a row before anyone noticed, because a cron job with no output channel just... stops. Nobody was checking the Actions tab every morning. The fix wasn't a longer-lived token. It was making failure loud: - name: Alert on failure if: failure env: GH TOKEN: ${{ secrets.GITHUB TOKEN }} REPO: ${{ github.repository }} RUN URL: ${{ github.server url }}/${{ github.repository }}/actions/runs/${{ github.run id }} run: | gh label create autopilot-failure --repo "$REPO" --color d73a4a \ --description "Content Autopilot run failed" 2 /dev/null || true body=$ printf 'Content Autopilot run failed: %s\n\nMost common cause: CLAUDE CODE OAUTH TOKEN expired or was revoked subscription tokens rotate . Re-sync the secret from a fresh token, or run claude setup-token for a long-lived one.\n\n@jonesrussell' "$RUN URL" existing=$ gh issue list --repo "$REPO" --label autopilot-failure --state open --json number --jq '. 0 .number // empty' if -n "$existing" ; then gh issue comment "$existing" --repo "$REPO" --body "$body" else gh issue create --repo "$REPO" --title "Content Autopilot is failing" \ --label autopilot-failure --body "$body" fi This runs on any step failure in the job, not just the auth step. It opens one issue, reuses it on repeat failures instead of spamming a new one per day, and GitHub's default notification settings turn that issue into an email. The permissions block needed issues: write added alongside the existing contents: write and id-token: write for this to work. Auth failures are loud and easy to fix. A boring, over-hedged, cliche-riddled post is a quieter failure, and nothing upstream of publish was checking for it. The next addition borrows the Self-Refine technique from the AI-writing research: have the model critique its own output once, then revise, before anything ships. Critique and then revise the blog post at <path and its social copy. Rubric: 1 voice matches docs/blog-style.md and the essay reference; 2 every claim is backed by a concrete fact, number, or reference; 3 sentence lengths vary no runs of same-length sentences ; 4 zero phrases from the site's banned AI-cliche list; 5 intro scope line is not the phrase 'This post covers'. The step is continue-on-error: true . If the critique call itself fails or times out, the workflow falls back to publishing the un-refined first draft rather than blocking the whole run over a quality pass. A revise pass helps, but it's still the same model marking its own homework. The step after it is deliberately dumb: scripts/slop-check.mjs runs no API call at all. It strips frontmatter, code fences, and markdown syntax, splits what's left into sentences, and scores three signals. js const mean = totalWords / lens.length; const variance = lens.reduce a, n = a + n - mean 2, 0 / lens.length; const stdev = Math.sqrt variance ; const burstiness = stdev / mean; // human ~0.6-1.2, AI slop < 0.4 Human writing varies sentence length a lot; AI writing tends to cluster around one length. The script also counts em dashes per thousand words over 20 reads as machine-written and matches a growing list of stock corporate-blog phrases pulled straight from the site's own style guide. It fails the draft on a hard burstiness floor of 0.32. Three or more banned-phrase hits alone are enough too, and so is any two of the three signals firing together. A failed check holds the draft: nothing gets committed, and the source issue stays open for the next day's run to try again. Build order matters here. The slop gate runs before the Hugo build gate, so a flagged draft never even reaches hugo --gc --minify , let alone git push . Two posts have shipped through this pipeline since it went live, tracked by issue number in data/autopilot-ledger.json . None of the hardening above came from planning ahead. Each piece exists because a specific run failed in a specific way: out-of-balance API credit, then a revoked token nobody caught, then no evidence-based reason to trust an unreviewed draft's prose. Unsupervised doesn't mean untested. It means every failure mode has to turn into a gate before the same thing is allowed to happen twice. Baamaapii