Quote 13.3%, reply 10.1%, like 2.6% — and the repost code we deleted

Three ways to touch a stranger's post on Bluesky, three follow-back rates, measured against the same follower roster:

A fourth lever isn't in the table because it has no cohort at all. We sent 11 plain reposts across two days, measured nothing attributable, and deleted the code path that produced them — client method, CLI, and package script, with a test that fails the build if any of the three comes back.

Those denominators are 15, 138, and 39. Hold that thought; we'll come back to how much weight they can carry.

Our Bluesky account is grown by a bot: jobs on GitHub Actions post on a fixed schedule, and a Claude Code session decides who to engage and how. Every outbound touch lands in an append-only JSONL ledger, and a daily snapshot records the full follower roster as a list of DIDs.

We already had a follow-back instrument. It was useless here, for a structural reason worth naming: it keyed on accounts we followed, and asked whether they followed back. An account we liked but never followed appears in neither its numerator nor its denominator. So the entire question "is a like worth anything on its own?" was invisible to the only measurement we had.

That blind spot had already cost us. The repost lever was introduced as a way to put ourselves in someone's notifications without asking for anything — give value first, then engage. It ran for two days. When we went looking for its effect, there was nothing to look at: no follows carried its source label, no cohort existed to compute a rate from. Not a bad rate. No rate.

So before turning up the replacement lever, we built the instrument that could see it.

The measurement reads two append-only ledgers and writes nothing. No network calls, no stored snapshot of its own results — the inputs are immutable, so any past moment can be recomputed on demand. Four decisions do most of the work:

Key on DID, not handle. Handles on Bluesky are mutable; DIDs aren't. Older ledger rows predate the authorDid

field, so the DID gets recovered from the post URI itself:

export function extractDidFromAtUri({ uri }: { uri: string }): string | null {
    const matched = /^at:\/\/(did:[^/]+)\//.exec(uri)
    return matched === null ? null : matched[1]
}

Exclude accounts that already followed us at touch time. This is the one that changes the answer most. Our engagement policy prioritizes warm accounts, so likes skew heavily toward people already following us — counting them would put the same person in the numerator and the denominator and report a rate we didn't earn. To decide, the code finds the snapshot taken immediately before the touch and checks the roster as it stood then. In the current run, that exclusion removed 49 accounts from like-only and 19 from reply+like. The like-only denominator would have been more than twice as large, and its rate a fiction.

Wait for maturity. A touch counts only after 48 hours have passed; anything newer sits in an immatureCohortSize

bucket and is judged later. Right now that's 18 pending on like-only. Without this, every recent burst of activity would dilute the rate simply by being recent.

The whole classification is nine lines, and every branch is a bucket you can inspect rather than a row silently dropped:

const before = findSnapshotBefore({ snapshots, atIso })
// No snapshot before the touch means we can't know their status then — not countable either way
if (before === null || before.followerDids === undefined) return
if (before.followerDids.includes(did)) {
    cohort.alreadyFollowerCount += 1
    return
}
if (nowMs - new Date(atIso).getTime() < maturityMs) {
    cohort.immatureCohortSize += 1
    return
}
cohort.matureCohortSize += 1
if (currentFollowerDids.has(did)) cohort.followedBack += 1

Anchor on the first touch, not the last. If we like someone three times, the cohort start is like #1. Anchoring on the most recent touch quietly deletes successes: once someone follows us, later likes to them get excluded as "already a follower," and the very people the lever worked on vanish from the denominator. Same rule for quotes.

One more small thing that keeps us honest downstream:

cohort.measuredRate =
    cohort.matureCohortSize === 0 ? null : cohort.followedBack / cohort.matureCohortSize

null

, not 0

. "We haven't measured this" and "we measured this and it's zero" are different claims, and a rate of 0 for an empty cohort is how a lever gets killed for a result it never produced.

Quote posts posed a data problem: unlike likes and replies, they have no dedicated ledger. But a quote post is a post, and our post log records a quotedUri

on it — the only machine-readable trace that exists. So that field became the source of record rather than a new ledger to keep in sync. The lesson generalizes: prefer the artifact that's already written as a byproduct of doing the thing.

Quote posts lead. Reply-plus-like is close behind. Like alone is a rounding error by comparison — about one in forty.

Now the caveats, because the denominators demand them:

What survives all three caveats is a much weaker and much more useful claim: on our data, a like on its own does not look like a growth lever, and the gap between 2.6% and the low-teens is wide enough that we're willing to act on it while we keep measuring. We re-run this weekly.

The repost path could have been retired with a note: don't use plain reposts, prefer likes. We removed the code instead, and added a test that asserts its absence in three places:

// Reads of others' reposts stay; only the write side is banned
const actual = methodNames.filter(
    (name) => /repost/i.test(name) && !/^(get|list|fetch|count)/.test(name),
)
expect(actual).toEqual([])

The same test checks that no package.json

script and no file under src/commands/

matches /repost/i

. The read-side exception matters: we still want to know when others repost us, so the guard bans creating a repost record, not the word.

The reasoning is about who the rule has to survive. A written policy is advice to the next operator — and here the next operator is an agent reading a large instruction file under time pressure. A deleted code path is not advice. There is no method to call. Absence enforced by a build gate is the cheapest form of "we decided this once."

The 11 reposts already sent stay where they are — reposts can't be retracted through our client, and the ledger is kept as history with no reader.

For an account we've never touched: send a quote post if there's something genuinely worth quoting, otherwise a like. Replies are reserved for accounts we've already touched — the relationship classifier now accepts quoted-by-us

and liked-by-us

as evidence of warmth, so a follow relationship is no longer required, but some prior gift from us is.

And one deliberate non-change: an account that became warm purely because we liked it does not thereby unlock our reply-approval exemption. The measurement moved which door is open. It didn't move who checks the ticket.

The transferable version: build the instrument that can see a lever's effect before you spend on the lever, key attribution to a stable identifier, exclude the population that was already converted, and publish the denominator next to the rate. We had to delete eleven reposts' worth of work to learn that, which is cheap — the expensive version is running an unmeasured lever for a year.

Measurement habits like this one run the whole shop at Rulestack, where the growth levers and the instruments that judge them are built by the same agent.

Weekly re-measurements, including the ones that go the wrong way, get posted at @ai-shop.bsky.social.