{"slug": "how-i-built-first-party-analytics-for-a-personal-blog", "title": "How I Built First-Party Analytics for a Personal Blog", "summary": "A developer rebuilt the analytics system for a personal blog to measure page views at the HTTP response boundary using Cloudflare Workers and D1, after discovering that a client-side JavaScript beacon miscounted AI reads as 97 when the actual number was lower. The new system logs server-side page observations, excluding prefetch requests and non-HTML content, to provide more accurate traffic data.", "body_md": "# How I Built First-Party Analytics for a Personal Blog\n\nA code-backed guide to first-party analytics for a personal blog—Cloudflare Workers, D1, public stats, privacy tradeoffs, and the wrong event that forced a rebuild.\n\nThe rebuild is deployed. The dashboard is live. The old data is still there.\n\nI still do not know if this was the right thing to spend my time on.\n\nA few hours into the rebuild, I stopped and asked:\n\n\"do you know why are we doing any of this at all? whats the goal\"\n\nThat question is still open.\n\nI want readers. I built a public analytics dashboard because I wanted to know whether anyone was arriving, which work found a path out of my own browser, and whether machines were requesting the agent-readable layer I had published.\n\nThen [the `llms.txt` investigation](/does-llms-txt-work) reached the number labeled **AI Reads**.\n\nThe number was 97. The query was correct. The claim was not.\n\nThe implementation is a first-party, cookieless system on Cloudflare Workers and D1. The underlying choice—client-side versus server-side analytics—starts with a smaller question: what exact event do you want one row to mean?\n\n## Why my JavaScript analytics beacon measured the wrong event\n\nThe old system began after a page loaded in a browser.\n\n``` php\nGET /article\n  -> static HTML bypasses application Worker\n  -> browser executes JavaScript\n  -> POST /api/event\n  -> classify the POST User-Agent\n  -> write one page_views row\n  -> count visitor_type = 2 as ai_fetches\n```\n\nA row meant: a client ran the page script, delivered a later analytics request, and D1 accepted the write.\n\nThat event was useful. It could count browser-rendered page events, derive a cleaned referrer hostname/path from `document.referrer`\n\n, estimate daily clients, and stay cheap because ordinary static requests never invoked my Worker.\n\nBut I used that row to support stronger nouns:\n\n**Page view** sounded like the original page request.**Visitor** sounded like a person.**Human** sounded verified.**AI Read** sounded like an AI system fetched and used the page.\n\nNone of those facts existed in the row.\n\nA normal crawler could request the HTML without executing JavaScript. A client could request `/llms.txt`\n\nor a page's Markdown representation directly. Those requests bypassed both the application Worker and the browser beacon.\n\n\"A correct counter can make the wrong claim more convincing.\"\n\n## Why copy and SQL could not fix client-side analytics\n\nThe tempting repair was smaller:\n\n- rename AI Reads;\n- add more crawler patterns;\n- fix the date window;\n- explain Visitors in a tooltip;\n- keep the architecture.\n\nThose changes would have improved the public language. They would not have made a non-JavaScript request observable.\n\nThe claim I wanted lived at the HTTP response boundary. Collection had to move there.\n\nThe Worker now handles application routes first, then serves the static asset once and gives the actual request and response to analytics:\n\n``` php\nGET /article\n  -> Worker routes known APIs or fetches the static asset\n  -> inspect the actual response\n  -> require successful, non-prefetch HTML\n  -> minimize metadata, classify the UA, derive the daily HMAC ID\n  -> waitUntil INSERT page_observations(source = edge)\n  -> aggregate one source-aware UTC contract for /api/stats\njs\nconst response = await env.ASSETS.fetch(request)\nobservePageResponse(request, response, env, ctx)\nreturn response\n```\n\nThe eligibility policy is deliberately narrow:\n\n```\nexport function isEligiblePageResponse(\n  request: Request,\n  response: Response,\n): boolean {\n  if (request.method !== 'GET' || !response.ok) return false\n\n  const path = new URL(request.url).pathname\n  if (path === '/stats' || path.startsWith('/stats/') || path === '/api' || path.startsWith('/api/')) return false\n  if (isPrefetch(request)) return false\n\n  const contentType = response.headers.get('Content-Type')\n  return contentType !== null && /^text\\/html(?:\\s*;|$)/i.test(contentType)\n}\n```\n\nOne new `PageObservation`\n\nmeans one recorded successful, non-prefetch HTML `GET`\n\n.\n\nIt does not mean the page rendered. It does not mean anyone read it. It does not mean the requester was a person. The D1 write is scheduled with `waitUntil`\n\n, so it remains best effort.\n\nThe event is smaller than the story I wanted. That is the point.\n\n## What changed across the analytics stack\n\nI expected to replace the beacon and adjust one query.\n\nInstead, the event definition reached through the whole system.\n\n| Layer | Before | Now | Why I chose it |\n|---|---|---|---|\n| Routing | Static pages bypassed application code; only `/api/*` ran Worker-first. |\nEvery request reaches the Worker, which serves the asset once and observes the actual response. | The original HTML request has to cross the instrument if Page view is going to mean a served page. |\n| Event | A row began with a later browser `POST /api/event` . |\nA row begins with an eligible successful HTML `GET` ; persistence remains best effort. |\nCopy and SQL cannot recover a request the sensor never saw. |\n| Vocabulary | Human, Visitor, and AI Read implied identity or cognition. | Browser, Daily client, Bot, AI UA, and Page view name observable or explicitly heuristic facts. | Public nouns should be the epistemic ceiling of the row. |\n| Referrer | JavaScript sent `document.referrer` ; ingestion stored a cleaned hostname + path. |\nThe original HTML request supplies its `Referer` header; the read model keeps the external hostname only. |\nAttribution now belongs to the event being counted. Host-level data answers channel decisions while reducing sensitive detail and fragmented rows. |\n| Identity | A public UTC date acted as the salt for a 64-bit SHA-256 prefix. | A Worker secret derives a daily HMAC key; D1 stores a 128-bit site/day client ID. | Keep within-day estimation while making offline candidate testing require the secret. It remains pseudonymous, not anonymous. |\n| Storage | `page_views` mixed old product words with nullable/defaulted fields. |\n`page_observations` constrains class, device, identity, owner, source, and timestamp; `page_views` stays the lossless archive. |\nPreserve history and provenance without pretending both collection methods mean the same thing. |\n| Time | The viewer's current offset shifted historical rows, and `30d` included 31 dates. |\nUTC half-open windows produce exactly 7/30/90 dates and zero-filled hourly/daily buckets. | The same public URL should return the same reproducible period for every reader. |\n| Aggregates | AI Reads ignored the active filter; owner predicates differed; chart fields changed granularity without changing language. | Every aggregate shares owner, traffic-class, path, and time predicates; both source eras remain visible. | One response should describe one population. `observation_source` preserves provenance rather than becoming a hidden filter. |\n| Presentation | New filter pills could temporarily sit above stale cards; failures left partial skeletons; units were implicit. | Loading, empty, failure, selected state, units, UTC range, accessible chart data, and methodology move together. | A mathematically correct response can still become a misleading interface. |\n| Privacy | Public copy said path/referrer only and fingerprint-free while more fields and a daily IP + UA identifier were stored. | The privacy page names transient inputs, stored fields, linkage, indefinite retention, legacy history, and separate Cloudflare Web Analytics. | Privacy is a threat model and disclosure contract, not the word cookieless. |\n\nThe public label was not decoration. It was the top of a data contract.\n\nChanging the noun without carrying the meaning through storage, queries, privacy, and presentation would have left another version of the same bug.\n\n## The Cloudflare Workers and D1 architecture\n\nThe Blog Worker owns HTTP routing and static delivery. The analytics package owns the meaning of a page observation. D1 stores source-aware facts. The stats service aggregates them but cannot reinterpret them into people, reads, or verified agents.\n\n``` php\nBlog Worker\n  -> eligibility policy\n  -> metadata minimization\n  -> traffic classification\n  -> daily client identity\n  -> observation repository\n\nStats request\n  -> query parser\n  -> UTC window policy\n  -> aggregate query service\n  -> zero-filled time series\n  -> shared public response contract\n```\n\nI used composition because those policies vary independently. Eligibility can change without changing HMAC construction. A new User-Agent rule does not need a new repository subtype. The Worker remains the composition root instead of becoming a superclass for analytics behavior.\n\nThe boundary also prevented a concrete dependency leak. My first shared `StatsResponse`\n\nexport pulled Worker-only D1 types into the browser's TypeScript program. Moving the public contract to a browser-safe `contracts`\n\nentry point restored the direction: the blog consumes analytics vocabulary without importing analytics infrastructure.\n\nThis is the useful part of domain-driven design here: split meanings and reasons to change, not files for their own sake. `PageObservation`\n\nowns served HTML. A future non-HTML `ResourceObservation`\n\nwould be a different context because its unit, privacy needs, retention, and product question are different.\n\n## How referrer attribution works now\n\nThe old beacon could not use the `Referer`\n\nheader on `POST /api/event`\n\n: that request came from the article to the same site, so its HTTP referrer described the current blog page. The browser had to send the navigation source explicitly:\n\n``` php\ndocument.referrer\n  -> beacon JSON body\n  -> parse URL\n  -> remove self-referrals and query/fragment\n  -> store hostname + pathname\n```\n\nThat path-level detail was useful. It could distinguish `reddit.com`\n\nfrom a specific Reddit thread or `github.com`\n\nfrom one repository page.\n\nThe edge model observes the original HTML request, so the request already carries the relevant `Referer`\n\nheader. The current policy reads it at the event boundary:\n\n```\nfunction referrerHost(request: Request, siteHostname: string): string | null {\n  const raw = request.headers.get('Referer')\n  if (raw === null) return null\n\n  try {\n    const host = new URL(raw).hostname.toLowerCase().replace(/^www\\./, '')\n    const selfHost = siteHostname.replace(/^www\\./, '')\n    return host === selfHost ? null : host\n  } catch {\n    return null\n  }\n}\n```\n\nThe read model keeps only the external hostname.\n\nI chose host-level attribution because the decisions I currently make are channel-level: did a reader arrive from Reddit, Google, Hacker News, X, GitHub, or ChatGPT? Host aggregation avoids splitting one channel into many path rows, stores less potentially sensitive detail on a low-traffic personal site, and removes a client-supplied field from the event body.\n\nThe tradeoff is real. I can no longer tell which exact Reddit thread, short link, or repository page sent a new edge observation. The untouched legacy archive still retains its cleaned hostname + path values; the public continuity copy normalizes them to hosts so the combined referrer list has one meaning.\n\nBrowser referrer policy, `noreferrer`\n\n, redirects, and privacy tools can still suppress or reduce the header. `null`\n\nmeans unattributed, not necessarily direct.\n\nThe tenet I am carrying forward is field-level: when an event moves, explain where every important field comes from now, why its granularity changed, and what decision justified the loss.\n\n## Cookieless analytics does not mean anonymous\n\nThe old identifier was the first 64 bits of:\n\n```\nSHA-256(UTC date + IP address + User-Agent)\n```\n\nThe date changed every day, but it was public and deterministic. Calling the result one-way did not make low-entropy inputs anonymous. A candidate IP and common User-Agent could still be tested offline.\n\nThe new edge identifier derives a daily key from a required secret, then signs a structured site/date/IP/User-Agent payload with HMAC-SHA-256. D1 stores the first 128 bits.\n\n```\ndaily key = HMAC(master key, UTC date)\nclient ID = HMAC(daily key, site + date + IP + User-Agent)[0..128]\n```\n\nRaw IP and raw User-Agent are not stored in the edge table. The identifier still links requests within one UTC day. It is still pseudonymous. It is not a person, and it is not anonymous.\n\nCloudflare's Worker temporarily sees the raw inputs. D1 retains the minimized observations without automatic expiration. The master key lives as a Worker secret, and I have not claimed a rotation schedule. Compromise of that boundary changes the enumeration threat; “no cookies” does not remove it.\n\nThat wording is less comfortable than “cookieless and fingerprint-free.” It is also closer to the system I built.\n\n## How I migrated five months of blog analytics without losing history\n\nMy first cutover plan was architecturally neat:\n\n- leave\n`page_views`\n\nas a legacy archive; - create\n`page_observations`\n\n; - start the public chart again from zero;\n- never mix the two event meanings.\n\nThen I inspected production.\n\nThe old table contained **2,564 rows** from March 7 through August 26:\n\n- 2,120 Browser-class beacon events;\n- 347 Bot-class beacon events;\n- 97 AI-UA-class beacon events;\n- 39 paths;\n- 32 referrer values;\n- 69 countries.\n\nThe data was not perfect. It was still meaningful.\n\nWhen I kept calling the two eras incomparable, I was solving the architecture more aggressively than the actual problem. This is my blog. I wanted its history.\n\nSo I changed the decision again.\n\nThe original table remains intact as the lossless source archive. A migration copies a minimized representation into the public read model and marks every copied row:\n\n```\nobservation_source <- beacon\nsource_event_id <- page_views.id\n```\n\nNew rows use `observation_source = 'edge'`\n\n.\n\nThe migration duplicates each legacy 16-hex daily ID to 32 hex characters. That preserves equality and distinct-count behavior; it does not add entropy or invent a stronger historical identity. It maps the old classes without inventing agent names, and a partial unique index makes the copy idempotent.\n\nProduction verification found:\n\n- source rows:\n**2,564**; - copied rows:\n**2,564**; - missing rows:\n**0**; - mismatched copied fields:\n**0**.\n\nThe public timeline now contains both eras. They are not perfectly comparable. The source marker and methodology say so.\n\nI chose useful continuity over single-era purity. That is not permission to combine arbitrary metrics. It is a judgment call whose provenance remains inspectable.\n\n## What client-side analytics was still better at\n\nMoving to the edge solved the event I cared about. It did not make the old design foolish.\n\nThe browser beacon had real advantages:\n\n- Static page delivery did not invoke application code.\n- Blocked JavaScript naturally stayed outside a browser-rendered metric.\n`document.referrer`\n\ndescribed browser navigation.- A\n`localStorage`\n\nflag could suppress my own beacon before it spent a Worker request or D1 write. - Browser timing and experience belong closer to Real User Monitoring than edge request analytics.\n\nThe edge system pays for broader visibility. Every static request now enters the Worker before the asset binding, even though only eligible HTML responses create D1 rows. Bot and AI classes still rely on sender-provided User-Agent strings. Writes remain best effort.\n\nServer-side is not universally better than client-side. It is better for the event this metric claims to represent.\n\n## Edge analytics versus Real User Monitoring\n\nThe two instruments answer different questions.\n\n| Edge PageObservation | Real User Monitoring |\n|---|---|\n| Was an eligible HTML response served? | What happened inside a real browser? |\n| Can observe non-JavaScript clients | Requires browser execution |\n| Sees HTTP method, path, status, media type, referrer, and request metadata | Sees rendering, Core Web Vitals, resources, interactions, and browser failures |\n| Cannot prove rendering or reading | Cannot see many direct crawler requests |\n\nCloudflare Web Analytics remains enabled as the separate RUM-like performance surface. My first-party D1 dashboard does not ingest it.\n\nI do not need to choose one instrument and pretend it answers both questions.\n\n## Why page analytics should not count llms.txt or Markdown\n\n`run_worker_first: true`\n\nmeans the Worker now sees requests for:\n\n`/llms.txt`\n\n;`/llms-full.txt`\n\n;- page-level Markdown;\n`/posts.json`\n\n;- RSS;\n- HTML and ordinary assets.\n\nD1 intentionally records only eligible HTML as Page observations.\n\nThat means the rebuild still does not answer the exact non-HTML retrieval question that helped trigger it. This is not another forgotten branch. It is a bounded-context decision: a Markdown request is not a page view.\n\n[Cloudflare request analytics](https://developers.cloudflare.com/ai-crawl-control/reference/graphql-api/) can estimate successful requests to those paths, while AI Crawl Control provides a narrower crawler view. Sampling and sender-identity limits still apply. The exact gate remains in [the predecessor's measurement protocol](/does-llms-txt-work#how-to-test-llmstxt-by-stage). A separate `ResourceObservation`\n\nmodel earns engineering only if repeated requests cross that gate and the result changes a decision.\n\nI fixed one observation boundary. I did not create a universal traffic event.\n\n## Owner exclusion still needs server-side configuration\n\nThe query consistently excludes rows where `is_owner = 1`\n\n.\n\nBut implementation capability is not production evidence. During the article audit, the active Worker did not expose an `OWNER_IPS`\n\nbinding, and neither era contained rows marked as owner.\n\nSo I cannot currently prove that my own requests are excluded.\n\nThe public methodology used to say they were. I corrected that wording while writing this article: only rows marked as mine are excluded, and that marking depends on server-side configuration. The remaining limit is operational, not hidden behind the predicate.\n\n## What I am building toward\n\nI am not trying to reproduce Google Analytics on a smaller budget. The first audit of whether this counter can be trusted came a week later, in [Which AI Fetchers Send Which Headers, Measured on a Live Site](/which-ai-fetchers-send-which-headers), where the raw requests of six AI fetchers were compared with what the counter recorded.\n\nI want a public decision instrument for one personal publication:\n\n- enough reach data to know whether work leaves my own browser;\n- page and referrer evidence that can change content, discoverability, or distribution;\n- automation classes without calling a User-Agent match a read;\n- visible collection-method changes instead of a magically continuous chart;\n- a privacy model that names transient inputs, retained fields, linkage, and deletion honestly;\n- separate instruments for served HTML, browser performance, non-HTML resources, referrals, and actual reader contact;\n- a stop condition: if a metric cannot change a named decision, I do not need to build it.\n\nThat vision explains why I spent the effort. The dashboard is public, the implementation is open, the historical source remains intact, and the method is inspectable. If I ask readers to trust a number, I want them to be able to see what became a row and where the claim stops.\n\nIt also limits the future work. The next steps are not “collect everything.” They are:\n\n- configure and verify server-side owner marking;\n- finish the declared 30-day comparison without changing the event mid-window;\n- keep using Cloudflare Web Analytics for RUM rather than duplicating it;\n- observe non-HTML resource demand through existing Cloudflare evidence before deciding whether\n`ResourceObservation`\n\ndeserves its own schema; - prefer replies, corrections, subscriptions, citations, and reports of use over adding another dashboard card.\n\nThe architecture is a bet that a smaller, explicit model will teach me more than a large system with convenient nouns.\n\n## An event-first checklist for trustworthy analytics\n\n### Name the event and its strongest noun\n\nWrite the event as a sentence:\n\n``` php\nactor\n  -> action\n  -> eligibility boundary\n  -> observation point\n  -> persistence point\n  -> strongest supportable public claim\n```\n\nIf the claim needs an event the instrument cannot observe, weaken the claim or move collection.\n\n“Human,” “visitor,” “read,” and “AI” add identity or intent. If the row does not contain evidence for that addition, the word is a data bug with good typography.\n\n### Split meanings, not files\n\nCreate a bounded context when two questions use different nouns, retention, privacy, or reasons to change. Page observations, non-HTML resource requests, RUM, referrals, and reader contact are related. They are not one analytics event with optional columns.\n\n### Preserve method changes as data\n\nA migration between measurement systems changes row meaning. Keep the source archive. Mark the read model with the collection method. Make the copy idempotent. Preserve unknowns instead of backfilling confidence.\n\n### Privacy is a threat model\n\n“No cookies” is not enough. Name the input, linkage window, stored fields, secret boundary, retention, and attacker. Call pseudonymisation what it is.\n\n### Every metric needs a decision\n\nThe allowed outcomes for this blog are small:\n\n- correct content;\n- improve discoverability;\n- change distribution;\n- run a bounded experiment;\n- do nothing.\n\nIf a metric cannot change one of those decisions, I do not need it.\n\nThe remaining question is allocation.\n\nI still do not know whether rebuilding this instrument was the right use of attention. I repaired a public trust problem. I also spent hours on analytics for a publication whose product is writing and contact with readers.\n\nA frozen 30-day continuation will compare a clean edge window with the historical beacon era, track cost and failures, and keep HTML observation separate from RUM and non-HTML requests.\n\nRight now I have a more honest event, 2,564 preserved rows, an owner-marking mechanism I still need to configure, and a question no dashboard can answer:\n\nWas repairing this instrument the right thing to do for a publication whose product is writing?\n\nIf you have built a precise counter for the wrong event, tell me where you found the mismatch. I am [@GogaKoreli](https://x.com/GogaKoreli).\n\n## Evidence ledger\n\n**Implementation and live evidence checked:** August 26, 2026. Provider behavior and production totals are volatile; event boundaries, source lineage, and decision rules are the durable layer.\n\n| Term / claim | Source | Evidence date |\n|---|---|---|\n| Original beacon could not observe direct static-resource requests |\n|", "url": "https://wpnews.pro/news/how-i-built-first-party-analytics-for-a-personal-blog", "canonical_source": "https://gkoreli.com/first-party-analytics-for-a-personal-blog", "published_at": "2026-08-26 00:00:00+00:00", "updated_at": "2026-09-03 06:53:16.644791+00:00", "lang": "en", "topics": ["developer-tools", "ai-infrastructure"], "entities": ["Cloudflare Workers", "D1"], "alternates": {"html": "https://wpnews.pro/news/how-i-built-first-party-analytics-for-a-personal-blog", "markdown": "https://wpnews.pro/news/how-i-built-first-party-analytics-for-a-personal-blog.md", "text": "https://wpnews.pro/news/how-i-built-first-party-analytics-for-a-personal-blog.txt", "jsonld": "https://wpnews.pro/news/how-i-built-first-party-analytics-for-a-personal-blog.jsonld"}}