# I paginated by 100 and lost 39 of 422 rows. At 99 and 101, nothing was missing. > Source: > Published: 2026-08-22 02:15:13+00:00 *This is a submission for DEV's Summer Bug Smash: Clear the Lineup powered by Sentry.* I wanted one number: what share of the entries in this challenge declare a particular prize category. It decides whether writing another entry is worth the evening. I got that number wrong three times in one morning. Every wrong answer looked finished. **First wrong answer: 19.** The listing endpoint returns titles and tags but not bodies, and the prize category is declared by a heading *inside* the body. So I counted titles containing "gemini" or "google". That found 19 of the 66 entries that actually carry the heading — and one of the 47 it missed was my own most recent entry, whose title mentions neither word. I had been quoting that proxy for days. **Second wrong answer: 23.9%.** So I fetched all 412 bodies, six at a time, no delay. My loader was this: ``` arts = [] for f in dir.glob("*.json"): try: a = json.loads(f.read_text()) if a.get("body_markdown"): arts.append(a) except Exception: bad += 1 ``` 277 of the 412 responses had the body `Retry later` . Two words, plain text, where JSON was expected. The `except` branch counted them and moved on, and I printed a clean table from 134 items: ``` items coverage category share naive 134 32.5% 23.9% complete 411 100.0% 18.5% ``` 5.4 points off, with nothing anomalous in it. **The number was not noisy. It was wrong, and it was plausible** — which is the only combination that actually costs you anything. The third wrong answer is the interesting one, and I only found it because I asked something else to look at my fix. That's further down. The loader has no bug in it. It does exactly what it says. The bug is in what it *means*: an exception during parse was being treated as evidence about the item, when it was evidence about the transport. ``` python def collect_strict(server, ids, max_rounds=8): """Same fetch, two extra rules: unparsed is not absent, and slow down on refusal.""" done, pending, rounds = {}, list(ids), 0 while pending and rounds < max_rounds: rounds += 1 still = [] for i in pending: rec = parse(server.get(i)) if rec is None: still.append(i) else: done[i] = rec pending = still server.cooldown //= 2 # back off: fewer requests per unit time missing = [i for i in ids if i not in done] if missing: raise RuntimeError(f"{len(missing)} of {len(ids)} ids never resolved") return [done[i] for i in ids], rounds ``` Against the real API — 0.45s between requests, doubling to a 2s ceiling — that recovered all 277 in four rounds: **134, then 77, then 63, then 3, then zero unresolved**. One id stayed unresolved and turned out to be a genuine `404` (an entry deleted between the listing call and the fetch), so the assertion permits *resolved-as-404* alongside *parsed*. The offline reproducer at the end of this post runs the same two collectors against a fake server that refuses in bursts: ``` items coverage in category share ground truth 412 100.0% 97 23.5% naive collector 157 38.1% 39 24.8% strict collector 412 100.0% 97 23.5% ``` Note how boring the naive row is. 24.8% against a true 23.5%. Nobody looks twice at that. With the numbers in hand, I handed the whole thing to **Gemini** and asked, among other things: *what can still silently truncate the dataset underneath my completeness assertion?* Its first-ranked answer was that my assertion validates the fetched set against **the listing**, and says nothing about whether the listing is complete. I went to check. This is the same endpoint, same tag, same minute, walked page by page until an empty page: ``` per_page pages unique ids note 25 17 422 empty page 18 50 non-JSON at page 10: 'Retry later' 75 6 422 empty page 7 99 5 422 empty page 6 100 4 383 empty page 5 101 5 422 empty page 6 125 4 422 empty page 5 150 3 422 empty page 4 200 3 422 empty page 4 ``` **Only per_page=100 loses data.** 99 is fine. 101 is fine. 100 comes up It gets better. Those four pages of 100: ``` rows returned : 400 unique ids : 383 ids appearing on more than one page: 17 id 4267457 appears 2x on pages [2, 3] id 4228478 appears 2x on pages [2, 3] ... ``` Four pages of 100 returned exactly 400 rows — the count you would sanity-check against — while containing **383 articles, 17 of them twice, and 39 not at all**. The 39 are not random: the `per_page=100` walk reaches back to 2026-07-14, the full listing reaches back to 2026-06-21. **It is the oldest entries that vanish**, which is precisely the population you would use to say anything about how the challenge has changed over time. I want to be careful about what I am claiming here. I did not find the cause inside DEV's code; I have no access to it. What I measured is that **the same query returns 422 or 383 items depending only on page size, and the short answer terminates cleanly.** That is reproducible from any machine, in about sixty requests, with no credentials. ``` python def assert_listing_stable(fetch, tag, sizes=(99, 100, 101, 200)): """A listing you cannot reproduce at two page sizes is not a population.""" counts = {} for pp in sizes: ids, page = set(), 1 while True: rows = fetch(tag, pp, page) if not rows: break ids.update(r["id"] for r in rows) page += 1 counts[pp] = ids best = max(counts.values(), key=len) for pp, ids in counts.items(): if len(ids) != len(best): raise RuntimeError( f"per_page={pp} yields {len(ids)} ids, per_page=" f"{max(counts, key=lambda k: len(counts[k]))} yields {len(best)}" ) return best ``` Nine lines, and it fails the build on an API defect I could not have guessed at. The offline reproducer is self-contained and needs nothing but the standard library. It builds a corpus whose property-of-interest is correlated with position — which is the case that matters — and runs both collectors against a server that refuses in bursts. ``` """A rate limiter that answers with the words "Retry later" is not an error your parser will notice. It is a sampler you did not know you installed. Runs offline against a fake server, so the numbers are reproducible without touching anyone's API. collect_naive() drops anything that fails to parse and returns what it got collect_strict() treats "did not parse" as "not fetched yet", slows down, retries, and refuses to return until every id is accounted for Requires: nothing but the standard library. """ import sys if hasattr(sys.stdout, "reconfigure"): sys.stdout.reconfigure(encoding="utf-8", errors="replace") import json N = 412 # items in the listing BURST = 8 # requests served before the limiter trips COOLDOWN = 16 # requests refused before it serves again def truth(i): """Ground truth: does item i belong to the category being counted? The first third of the listing is denser than the rest -- newer submissions mention the prize category more often than older ones. That is the part that matters: the property being counted is correlated with position. """ threshold = 40 if i < N // 3 else 15 return (i * 37) % 100 < threshold class Server: """Serves JSON in bursts. When the limiter trips it answers 'Retry later'. The refusal is a 200 with a plain-text body. Nothing raises, nothing retries itself, and the caller gets a str where it expected JSON. """ def __init__(self, cooldown=COOLDOWN): self.n = 0 self.cooldown = cooldown self.served = 0 self.refused = 0 def get(self, i): self.n += 1 if self.cooldown and (self.n % (BURST + self.cooldown)) > BURST: self.refused += 1 return "Retry later\n" self.served += 1 return json.dumps({"id": i, "in_category": truth(i)}) def parse(raw): try: return json.loads(raw) except Exception: return None def collect_naive(server, ids): """What I actually wrote. There is no bug in it -- it does exactly what it says.""" out = [] for i in ids: rec = parse(server.get(i)) if rec is not None: out.append(rec) return out def collect_strict(server, ids, max_rounds=8): """Same fetch, two extra rules: unparsed is not absent, and slow down on refusal.""" done, pending, rounds = {}, list(ids), 0 while pending and rounds < max_rounds: rounds += 1 still = [] for i in pending: rec = parse(server.get(i)) if rec is None: still.append(i) else: done[i] = rec pending = still server.cooldown //= 2 # back off: fewer requests per unit time missing = [i for i in ids if i not in done] if missing: raise RuntimeError(f"{len(missing)} of {len(ids)} ids never resolved") return [done[i] for i in ids], rounds def share(records): n = len(records) hits = sum(1 for r in records if r["in_category"]) return hits, n, (hits / n * 100 if n else 0.0) def main(): ids = list(range(N)) actual = [{"id": i, "in_category": truth(i)} for i in ids] naive = collect_naive(Server(), ids) strict, rounds = collect_strict(Server(), ids) print(f"{'':<20}{'items':>8}{'coverage':>11}{'in category':>14}{'share':>9}") for label, recs in (("ground truth", actual), ("naive collector", naive), ("strict collector", strict)): hits, n, pct = share(recs) print(f"{label:<20}{n:>8}{n / N * 100:>10.1f}%{hits:>14}{pct:>8.1f}%") _, _, t = share(actual) _, _, g = share(naive) print(f"\nThe naive collector reported {g:.1f}% from {len(naive)}/{N} items. " f"The answer is {t:.1f}%.") print(f"It raised nothing, logged nothing, and its table looked complete.") print(f"The strict collector resolved every id in {rounds} rounds.") if __name__ == "__main__": main() ``` The page-size sweep is fifteen lines of `curl` around the same idea and is quoted in full above. **Done:** `assert_listing_stable` runs before the fetch. On this tag it fails, correctly.**Done because the review said so:** **Not done, and I'd rather say so:** `curl -o` and `resp.raise_for_status()` would have caught the whole thing is one I destroyed the evidence for. Sequential requests do not reproduce it, and I am not going to hammer a free API until it stops talking to me just to find out. The lesson stands on its own: **Left as a guard, deliberately:** Recomputing the denominator at two page sizes every time, even when nothing has changed. It costs about sixty requests and it is the only reason I know the number at all. I used **Gemini** (free tier, Flash) once: after the fix worked, with the numbers already in hand, to ask what my fix still could not see. I gave it both wrong answers, the loader, the strict collector, the assertion, and four questions. I did not adopt any of it — I measured each claim. It named the class first: This is a Silent Partial Sample(or Silent Truncation) resulting fromPlausible Degradation. [...] The system experienced soft failures—surrogate metrics (Attempt 1) and swallowed rate-limit payloads (Attempt 2)—that degraded the data quality into a plausible subset rather than triggering an explicit system error. That one sentence covers both of my wrong answers, which is what I had asked for and had not managed to write myself. Then four ranked failure modes, and a verdict on the fix. ``` Claim Verdict Measurement Rank 1 the listing itself is truncated held per_page=100 -> 383 of 422, 17 duplicated rows, empty page 5 Rank 2 valid JSON that lacks the key fields held the deleted entry parses fine and has no body_markdown Rank 4 HTTP 200 carrying an error payload not observed the deleted entry returns a real 404 Q3 backoff+assert is a patch, not a fix accepted and see "Not done" above Q4 I may not call the 32.5% sample biased conceded claim retracted, see below ``` **Rank 1 is the reason this post exists.** I asked what my completeness check could not see, and the answer was: the thing it checks against. Sixty requests later I had an API defect that is reproducible by anyone, in which the page size everybody reaches for first is the only one that loses data. **Question 4 cost me a claim I liked.** I had written that the 32.5% sample was *biased*, not merely small, because rate-limit refusals arrive in bursts and bursts land on neighbours in a listing. Gemini pointed out that I overwrote the failed responses with the successful refetch, so I no longer have the evidence for that: You cannot claim the 32.5% sample was definitively biased due to listing-order burst clustering. You cannot claim spatial, chronological, or network locality for the failed items, because you lost the exact temporal sequence and per-request metadata. What it left me is narrower and I think stronger: The 32.5% sample is unvalidated and methodologically unreliable because the sampling mechanism was governed by server load-shedding rather than random selection. [...] the sample cannot be assumed to be Missing Completely at Random (MCAR). So: 23.9% versus 18.5% is an empirical divergence of 5.4 points, and the sample was drawn by a server deciding what it felt like answering. That is enough to throw the number away. It is not enough to say *why* it leaned the way it did, and I have edited that claim out of my notes. One last thing, which happened while I was measuring the above. My sweep script crashed decoding `curl` 's output — the console here is cp932 and one of the titles was not — and my error handler, which was written to notice `Retry later` , reported it as **"rate-limited at page 1"** for all nine page sizes. I spent a minute believing the API had cut me off, in the middle of writing a post about mistaking a local failure for an absent record. That is the whole thing, really. Six of my submissions this month are the same shape: *every check passed and the answer was still wrong.* A ranking only I could see. A collector reporting success on 27% of the data. A rate limiter counting the retries. A verifier that read every character back while three links in the document pointed at nothing. And now a denominator that was wrong three ways, where the third way was invisible until I asked something outside my own head what my check was not looking at.