If you let AI write your posts, what you get is an endless stream of articles that look fine but leave nothing behind once you've finished reading. This is part of my ongoing series on building automation infrastructure for shipping personal projects at scale with Claude Code. This time I want to talk about the "5% human check" gate I designed to keep quality high while mass-producing long-form X (formerly Twitter) Articles with Claude Code.
When you let AI write, you quickly end up mass-producing articles that look fine but leave nothing behind once you've finished reading. My honest take is that what determines the post-read impression isn't the accuracy of the information, but whether there's any firsthand information in it.
At first I aimed for a simple three-stage pipeline: generate → quality check → auto-post to X. Wire it all together in shell scripts and run it on cron. Technically it isn't hard. But when I actually ran it, all it did was quietly post uniform 7-out-of-10 articles. You could learn something from them, but they were faceless — "who even wrote this?" It didn't work as something published under Lily's name.
The cause was clear. Claude is good at "aiming for the average score." It tends to produce accurate but stimulation-free articles that offend no one. The conclusion I reached: to ship a 9/10 article, a human has to hold the final gate.
So I settled on a design where Claude mass-produces, and a human touches only 5% to pick out the 9/10s. I gave up on full automation and focused on compressing human judgment down to the minimum amount of work.
Here's the directory structure of ~/dev/x-article-factory/
.
x-article-factory/
├── generate.sh # Have Claude draft, then self-score
├── review-gate.sh # Human approves/rejects in a TUI
├── daily.sh # launchd wrapper (drafts 2 per day)
├── knowledge/
│ ├── persona.md # Lily's persona and CTA
│ ├── style.md # Voice and tempo rules
│ ├── post-types.md # Usable formats (How-to / failure story / before-after, etc.)
│ ├── quality-rubric.md # Criteria for the 10 self-scoring items
│ └── hooks.md # Collection of opening-hook patterns
├── drafts/ # Drafts that passed the quality gate
├── approved/ # Human-approved articles (awaiting posting)
├── rejected/ # Human-rejected articles
└── logs/
├── gen.log # Generation log (OK / REJECTED_LOWSCORE)
├── posted-types.log # History of formats used (last 3 used for dedup)
└── posted-topics.log # History of topics used (last 8 used for dedup)
The processing flow has three stages.
[launchd] daily.sh
↓ Loop N articles (default 2)
generate.sh
↓ Claude drafts → self-scores (only avg ≥ 7.0 passes)
drafts/YYYY-MM-DD_HHMMSS.md
↓ bash review-gate.sh (human decides in a TUI)
approved/ ← only the 9s get through
rejected/ ← even 7–8s get dropped if they don't land
↓ Manually paste into X's scheduled posts
The division of labor is simple: AI generates and self-scores, the human just makes the final call.
The heart of generation is generate.sh
. Let's walk through the design in order.
MODEL="${X_ARTICLE_MODEL:-sonnet}"
GEN_TIMEOUT="${X_ARTICLE_GEN_TIMEOUT:-420}"
MIN_AVG="${X_ARTICLE_MIN_AVG:-7.0}"
The model defaults to sonnet
, the timeout to 420 seconds (7 minutes), and the quality threshold to 7.0. Drafting long-form articles takes time to generate, so I set the timeout with plenty of headroom. Everything can be overridden via environment variables, so experiments like "raise the threshold to narrow down quality" or "try a different model" can be toggled with a single line in .env
.
There's also a mechanism where the script exits immediately if a D
file exists.
[ -f "$DIR/D" ] && { echo "[gen] D。停止中。"; exit 0; }
If things go off the rails or unexpected articles start coming out, I can stop everything with just touch ~/dev/x-article-factory/D
.
KNOW=""
for f in persona hooks style post-types quality-rubric; do
KNOW+=$'\n\n===== '"$f"$' =====\n'"$(cat "$DIR/knowledge/$f.md")"
done
I manage the instruction set for Claude across five files. Here's the role of each.
| File | Contents |
|---|---|
| persona.md | Lily's persona settings, exit-CTA patterns, publishing don'ts |
| style.md | Rules for voice, tempo, and persuasiveness (including concrete phrasing) |
| post-types.md | List of usable formats (numbered; format name, structural pattern, example fitting topics) |
| quality-rubric.md | Scoring criteria for the 10 self-scoring items (the 1–10 definition for each) |
| hooks.md | Collection of opening-hook patterns (question type, shocking-fact type, before-after type, etc.) |
Rather than one giant prompt, splitting management across files made fine tuning easier — "update only style.md" or "change only the CTA wording."
This is the single most important point of the factory design.
GROUND=""
for src in "$HOME/.remember/recent.md" "$HOME/.remember/now.md" \
"$HOME/Documents/claude-obsidian/wiki/hot.md"; do
[ -f "$src" ] && GROUND+=$'\n\n--- '"$(basename "$src")"$' ---\n'"$(head -c 4000 "$src")"
done
[ -z "$GROUND" ] && GROUND="（作業ログ無し。一般的なClaude Code/個人開発の知見で書く）"
I load three sources as the "firsthand information" passed to Claude.
~/.remember/recent.md
— recent work notes~/.remember/now.md
— what I'm working on right now~/Documents/claude-obsidian/wiki/hot.md
— the hot summary from Obsidian (about 500 words)Each is read up to 4,000 bytes with head -c 4000
and passed into the prompt. The instruction in the prompt is: "Always pick up proper nouns, numbers, and failures and put them into the body. Don't fall back on borrowed generalities." That two-part combination does the work.
For example, if grounding contains information like "the iOS app passed review," "an error came up in a certain step of a script," or "the impression count of a feature I shipped this week," Claude will try to make that the protagonist of the article. This is where the distance from AI mass-produced junk is created.
When all files are empty, it falls back to "no work log" and writes based on general knowledge. Quality drops, so the habit of updating your work log before generating an article is important.
LAST_TYPES=$(tail -3 "$TYPES_LOG" | paste -sd '、' -); [ -z "$LAST_TYPES" ] && LAST_TYPES="（まだ無し）"
LAST_TOPICS=$(tail -8 "$TOPICS_LOG" | paste -sd '、' -); [ -z "$LAST_TOPICS" ] && LAST_TOPICS="（まだ無し）"
I pass Claude the last 3 formats and the last 8 topics. By pairing this with instructions like "avoid these and choose a different format" and "avoid rehashing; take a different angle," I prevent runs of articles with the same pattern.
The reason topics get a wider window (8) than formats is that topic exhaustion comes first. There are more than 10 formats, but the "Lily lane" of topics (Claude Code / personal dev / AI automation) is narrow.
I make the output format specification for Claude strict.
1行目: TYPE: <使った型の名前（post-types.mdの番号と名前）>
2行目: TOPIC: <この記事のテーマを15字以内で>
3行目: SCORE: {"hook":N,"value":N,"specificity":N,"firsthand":N,"tempo":N,"reproducibility":N,"structure":N,"surprise":N,"style":N,"cta":N,"avg":N.N}
4行目以降: 記事本文のMarkdown（1行目は「# タイトル」）
The SCORE JSON contains 10 items. Claude scores the article it wrote itself along 10 axes.
| Item | Scoring perspective |
|---|---|
| hook | The pull of the opening hook. Does the reader feel like reading on? |
| value | Substantive value delivered to the reader. Does reading it change anything? |
| specificity | Concreteness / density of proper nouns. Has it slipped into abstraction? |
| firsthand | Presence of firsthand information. Is it based on real work or borrowed knowledge? |
| tempo | Tempo and readability. Does scrolling never stop? |
| reproducibility | Reproducibility. Can readers try it themselves? |
| structure | Logical soundness of the structure. Is the flow forced anywhere? |
| surprise | Surprise / discovery. Does the reader feel "I didn't know that"? |
| style | Match with Lily's voice. Is it off from the persona? |
| cta | Naturalness of the CTA. Is it pushy? |
AVG=$(printf '%s' "$SCORE" | python3 "$DIR/lib/parse_avg.py" 2>/dev/null)
[ -z "$AVG" ] && AVG=0
if python3 -c "import sys; sys.exit(0 if float('$AVG') >= float('$MIN_AVG') else 1)" 2>/dev/null; then
:
else
echo "[gen] 品質スコア avg=$AVG < $MIN_AVG。棄却（drafts に出さない）。topic=$TOPIC" >&2
echo "$(date '+%F %T') REJECTED_LOWSCORE avg=$AVG $TOPIC" >> "$DIR/logs/gen.log"
exit 2
fi
It extracts avg
from the SCORE JSON and rejects anything below 7.0. When rejected, it doesn't output to drafts/
and records it in the log as REJECTED_LOWSCORE
.
The design intent is: "if Claude self-scores below avg 7.0, it never reaches human eyes in the first place." This drops the human review effort for 7-ish articles to zero. Looking at the logs, 20–30% of drafted articles get rejected at this gate.
Note:"Passing 7.0 = a good article" is not true. Claude's self-scoring tends to be lenient, and when you actually read them, there are plenty of "7.2 but doesn't land" articles. The 7.0 gate is a filter to knock out the obvious failures. Picking the 9/10 articles is the human's job.
Anything that passes the quality gate is written to drafts/
.
TS=$(date +%Y-%m-%d_%H%M%S)
FNAME="$DIR/drafts/${TS}.md"
{
printf -- '<!-- TYPE: %s | TOPIC: %s | SCORE_AVG: %s | gen: %s -->\n\n' "$TYPE" "$TOPIC" "$AVG" "$TS"
printf '%s\n' "$BODY"
} > "$FNAME"
The filename is a timestamp (2026-06-26_142030.md
format). The first line embeds meta info as an HTML comment. By displaying this meta line in review-gate.sh, the "format / topic / score" is visible at a glance during review.
resp_is_valid() {
local r="$1"
[ -z "$r" ] && return 1
printf '%s' "$r" | grep -q '^TYPE:' || return 1
printf '%s' "$r" | grep -q '^SCORE:' || return 1
printf '%s' "$r" | grep -qiE 'request timed out|usage limit|rate limit' && return 1
[ "$(printf '%s' "$r" | wc -c | tr -d ' ')" -lt 600 ] && return 1
return 0
}
This is the validation function for the generation result. It checks for the presence of the TYPE line, the presence of the SCORE line, the absence of timeout/rate-limit errors, and a minimum body size of 600 bytes. If any of these is missing, it's treated as invalid and retried up to 3 times.
for attempt in 1 2 3; do
RESP=$(timeout "$GEN_TIMEOUT" "$CLAUDE" -p "$PROMPT" --allowedTools WebSearch --model "$MODEL" </dev/null 2>/dev/null)
resp_is_valid "$RESP" && break
echo "[gen] 生成失敗(試行${attempt}/3)。再試行…" >&2
RESP=""
done
I add --allowedTools WebSearch
so that Claude can use WebSearch to "check what's current right now." The prompt instructs it to "check what's timely with WebSearch if needed," and it kicks in when picking up timely material.
The human reviews the articles in drafts/
that passed the quality gate, one at a time, in a TUI.
echo "未チェックの下書き: ${#drafts[@]} 本"
echo "操作: [a]承認 [s]保留 [d]却下 [e]編集 [q]終了"
There are five choices.
| Key | Action |
|---|---|
| a | Move to approved/ (enters the posting queue) |
| s | Keep in drafts (re-decide later) |
| d | Move to rejected/ |
| e | Open in $EDITOR , edit, then re-decide |
| q | Quit |
The preview shows the meta line (format / topic / score) plus the first 60 lines of the body.
head -1 "$f" | sed 's/<!-- *//; s/ *-->//' # メタ行（型/テーマ/スコア）
tail -n +2 "$f" | sed '/./,$!d' | head -60
The key point of the TUI is that it's designed so you don't have to read the whole thing. The intent is that the first-60-lines preview lets you intuitively judge "approve or not." Even if the score is 7.5 or higher, if reading it feels "not very Lily" or "the firsthand info is basically generalities," I hit d
and reject it immediately.
Conversely, even at a score of 7.2, if I feel "this episode lands," I sometimes open the editor with e
, make a small fix, and approve.
The output after review ends is simple.
echo "承認済み: $(ls "$DIR"/approved/*.md 2>/dev/null | wc -l | tr -d ' ') 本（approved/）"
echo "次: 承認分をXの予約投稿に貼る → 投稿したら approved/ から消す"
I manually paste approved articles into X's scheduled posts, and delete them from approved/
after posting. By running the approve → post → delete cycle, queue management is fully handled with just ls approved/
.
As a realistic sense of the pass rate, approving 1–2 articles out of every 10 generated (10–20%) is realistic. I run it on the hypothesis that "posting 1–2 nine-out-of-tens beats posting ten seven-out-of-tens for overall engagement."
N="${X_ARTICLE_DAILY_N:-2}"
export NVM_DIR="${NVM_DIR:-$HOME/.nvm}"
[ -s "$NVM_DIR/nvm.sh" ] && . "$NVM_DIR/nvm.sh" >/dev/null 2>&1
[ -f "$DIR/D" ] && { echo "[daily] D"; exit 0; }
ok=0
for i in $(seq 1 "$N"); do
if bash "$DIR/generate.sh"; then ok=$((ok+1)); fi
sleep 2
done
echo "[daily] $(date '+%F %T') 起草 $ok/$N 本 → 人間チェック: bash review-gate.sh"
This is the launchd wrapper script. By default it drafts 2 articles a day (changeable via X_ARTICLE_DAILY_N
).
What matters is nvm PATH resolution. launchd's minimal environment doesn't have nvm, and in its bare state you get claude: command not found
. Explicitly sourcing nvm.sh
resolves it. If you forget this, you end up in the confusing situation where it works in an interactive shell but fails only via launchd.
The sleep 2
in between is a stability measure. Rather than firing off N articles in parallel, waiting 2 seconds before firing the next one makes the API calls more stable.
Inside the loop, it watches generate.sh
's exit code to count successes (because when rejected at the quality gate it returns exit 2, it isn't added to ok
). In the end, the result is printed in a form like "drafted 1/2."
After actually running it for a while, several things came into view.
The quality-gate threshold of 7.0 is a realistic lower bound
I first tried 6.5, but a lot of the high-6 articles were "readable but thin," and they all became d
in review-gate.sh. After switching to 7.0, many of the ones that passed got closer to "interesting to read." On the other hand, raising it to 8.0 drops the pass rate too far, and drafts stop accumulating. 7.0–7.5 is the realistic operating range.
The richer the grounding files, the higher the density of the article
On days when Obsidian's hot.md is written up to 500+ words versus days when it's nearly empty, the concreteness of the article is clearly different. The habit of frequently jotting down "what I did today" and "the numbers that came out this week" directly translates to article quality. Writing a work log is also prep for the article.
Format history management (last 3) works for dedup suppression
Because it tended to produce the same format (e.g. "failure story") back to back, I instructed it to avoid the last 3. That alone increased format diversity. The reason topics look at the last 8 is that material runs out faster within the same lane. Topics run dry faster than formats.
The e (edit) option gets used more than you'd think
The e
key in review-gate.sh is "open in $EDITOR, edit, then re-decide." At first I thought "editing is a hassle so I'll barely use it," but in practice it's handy in situations like "the whole thing is good but I want to fix just the final CTA" or "I want to add just one sentence of firsthand info with my own number." The shape where the human adds 20% on top of the 80% foundation AI wrote tends to make the highest-engagement articles.
** claude: command not found from launchd** → nvm's PATH isn't set. Unless you source
[ -s "$NVM_DIR/nvm.sh" ] && . "$NVM_DIR/nvm.sh"
in daily.sh
, it works in an interactive shell but fails only from launchd. It ate a lot of my debugging time.Forgetting </dev/null and freezing on an interactive wait →
claude -p
can go into interactive mode if stdin isn't closed. It stays stuck for the full 420-second timeout before failing. Always close stdin with </dev/null
.A broken SCORE-line JSON causing a streak of AVG=0 all-rejects → Claude sometimes mixes newlines or code fences into the SCORE line.
parse_avg.py
fails to parse the JSON, AVG
becomes 0, and every article gets rejected. I needed to write the resp_is_valid()
validation strictly.A <!-- TYPE:-like string sneaking into the body and being misjudged as the meta line → Claude sometimes reads the output-format explanation and writes something resembling the meta line inside the article body. I solved this by limiting body extraction to everything after the first
# heading
using awk '/^# /{p=1} p{print}'
.A quality-gate pass rate so high it increases review effort → Even at a 7.0 threshold, because it "scores itself and passes itself," there are days when Claude scores leniently. Don't skip the human's final call in review-gate.sh. The gate is only a "filter for obvious failures," not a "guarantee of a 9/10."
Fallback articles with no firsthand info slipping through → When grounding is empty it falls back to "(no work log; write from general knowledge)," but articles written in this mode almost all become d
in review-gate.sh. Drafting without enriching grounding tends to end in wasted effort.
drafts/
.~/.remember/
files and Obsidian's hot.md) feeds in real work logs, making the article firsthand-information-based. This is where you create distance from AI mass-produced junk.Next time I plan to write about measuring the performance of approved, posted articles and feeding it back into the next generation — that is, the mechanism for logging impressions, like rate, and save rate, and using them for the next round of grounding.
*Written by Lily — I ship iOS apps and automate my content stack with Claude Code.
Follow along: Portfolio · X · GitHub*