{"slug": "analytics-from-nothing-but-nginx-logs", "title": "Analytics From Nothing But nginx Logs", "summary": "Developer Jake Marsh built a cookie-free blog analytics system that derives page views, referrers, bot traffic and feed-reader counts from nginx access logs, avoiding third-party analytics services such as Google Analytics and Plausible. The site's JavaScript sends one small beacon request per page view to a /hit path that nginx logs and answers with an empty response, and a Bun script, deallocated-stats.ts, runs via a systemd timer at 00:20 UTC daily to parse the current log plus the two most recent rotated logs, which are deleted after 14 days. The beacon omits credentials, skips firing for visitors sending Do Not Track or Global Privacy Control, and sends only the path and referring domain, while the log parser never captures IP addresses.", "body_md": "# Analytics From Nothing But nginx Logs\n\n[Jake Marsh](https://deallocatedobjects.com/about)\n\nI wanted to know a few things about this blog. Which posts people read, where they come from, how often the crawlers behind Claude, ChatGPT and Perplexity stop by, whether anyone fetches the markdown versions of posts, and how many people follow along in a feed reader. I didn’t want to set a cookie, load a script from someone else’s server, or keep anything about any one person to get there.\n\nIt turns out nginx already writes down almost everything I needed. The rest is one tiny request from the page, a short Bun script that runs once a day, and a report I run from my Mac.\n\n## Why Not Google Analytics or Plausible[#](#why-not-google-analytics-or-plausible)\n\nEvery hosted analytics product, including the privacy-friendly ones, means the page talks to another company’s server. That’s a third-party request on every page view, and it’s one more thing a reader would reasonably want to know about (or consent to). Everything on Deallocated Objects loads from deallocatedobjects.com, and I wanted to keep it that way.\n\nI also didn’t need much. I don’t need sessions, funnels or a dashboard with forty charts. I need counts, and counts are cheap to make from data the server already has.\n\n## The Problem With Logs Alone[#](#the-problem-with-logs-alone)\n\nServer logs are great for a lot of this, with one big catch. Most of the requests to a blog like this one come from bots. Search engines, the crawlers behind chatbots, link preview fetchers, feed readers and a long tail of scripts all show up as `GET /posts/something`, and plenty of them send browser-looking user agents. If I counted page views straight from the log, I’d mostly be counting crawlers.\n\nA request made by the page’s own JavaScript is different. It only happens when a real browser loads the page and runs its script. So the site’s script sends one small request per page view:\n\nIt carries the path and the referring site’s domain. Only the domain: a link from someone’s private Slack thread or a search results page shows up as the host and nothing else. `credentials: \"omit\"` means no cookies ride along (there aren’t any anyway), and `keepalive` lets the request finish even if you click away right away. If your browser sends Do Not Track or Global Privacy Control, the request never happens. It also only fires on the real domain, so my local previews don’t count.\n\n## The nginx Side[#](#the-nginx-side)\n\nThere’s no endpoint behind `/hit`. nginx logs the request like any other and returns an empty response:\n\nThe blog gets its own `access_log` so the rollup only has to read one site’s traffic, and that log rotates and gets deleted after 14 days like the rest of the box’s logs. The `no-store` header keeps any cache from answering the beacon on nginx’s behalf, which would make views disappear.\n\n## The Daily Rollup[#](#the-daily-rollup)\n\nA systemd timer runs `deallocated-stats.ts` at 00:20 UTC every day. It reads the current log plus the two most recent rotated ones (so a rotation at midnight doesn’t lose anything), keeps the lines from yesterday, and parses the standard combined log format with one regex:\n\nNotice the IP address is matched by that first `\\S+` and never captured, so it never ends up in a count or a file.\n\nEvery `/hit` line becomes one reader page view for its path, plus one referrer count if the referring domain isn’t this site. Every other line gets checked against a table of bot families by user agent:\n\nThe first match wins, so the catch-all “Other bots” row sits at the bottom. The same pass counts fetches of the agent-friendly files from my [last post](https://deallocatedobjects.com/posts/making-my-blog-readable-by-ai-agents): any `.md` URL, `llms.txt`, `llms-full.txt`, `feed.json` and `search.json`. It also tallies status codes and collects the paths that returned 404, skipping the endless `wp-login.php` and `.env` probes so the list only shows links that someone might have expected to work.\n\n## Counting Feed Subscribers[#](#counting-feed-subscribers)\n\nFeed readers fetch the feed on behalf of everyone subscribed to it, so one Feedbin fetch might represent a lot of readers. Most of the hosted ones say so right in their user agent, something like `Feedbin feed-id:1234 - 42 subscribers`. So the script just reads it:\n\nIt takes the highest number each reader reported that day, since a service fetches many times a day and the count shouldn’t add up across fetches. Readers that don’t report a number get counted as fetches instead, which still tells me they’re out there.\n\n## Counts Only[#](#counts-only)\n\nWhen it’s done, the script writes one small JSON file per day to `/var/lib/deallocated-stats/`: total requests, reader page views, top pages, referrers, bot families, agent file fetches, feed subscribers, status codes and 404s. Every value is a number next to a path, a domain or a name. No IP addresses, no user agents, nothing that could tie two requests to the same person.\n\nThat’s what lets the [privacy page](https://deallocatedobjects.com/privacy) make simple promises. The raw logs go away after 14 days, and the counts that outlive them never include an IP address. Keeping counts forever is fine because counts can’t be about anyone.\n\n## Why a systemd Timer[#](#why-a-systemd-timer)\n\nThe script runs as root because nginx’s logs aren’t readable by ordinary users, so I wanted it scheduled by something I can see and audit with `systemctl list-timers`. `Persistent=true` means if the box is down at 00:20, the rollup runs as soon as it’s back. The day it rolls up comes from an environment variable, so I can rerun any day by hand while the logs for it still exist.\n\nThere’s also no database and no running process. The timer wakes up, Bun runs one file for a moment, and it exits.\n\n## The Report[#](#the-report)\n\nOn my Mac, `bun scripts/stats.ts` pulls the last seven daily files over SSH and adds them together (`bun scripts/stats.ts 30` for a month, `--run` to roll up yesterday right now). It prints a header with the date range, total reader page views against total requests and a per-day line, then short ranked lists: top pages, referrers, bots and crawlers, agent-friendly files fetched, feed readers with their latest subscriber counts, and the top 404s marked “worth a redirect?” That last list is the handiest one, since a blog with posts going back to 2010 has plenty of old links pointing at URLs that moved.", "url": "https://wpnews.pro/news/analytics-from-nothing-but-nginx-logs", "canonical_source": "https://deallocatedobjects.com/posts/analytics-from-nothing-but-nginx-logs", "published_at": "2026-10-26 13:00:00+00:00", "updated_at": "2026-10-26 13:14:06.951077+00:00", "lang": "en", "topics": ["ai-crawlers"], "entities": ["Jake Marsh", "Deallocated Objects", "nginx", "Bun", "Google Analytics", "Plausible", "Claude", "ChatGPT"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/analytics-from-nothing-but-nginx-logs", "markdown": "https://wpnews.pro/news/analytics-from-nothing-but-nginx-logs.md", "text": "https://wpnews.pro/news/analytics-from-nothing-but-nginx-logs.txt", "jsonld": "https://wpnews.pro/news/analytics-from-nothing-but-nginx-logs.jsonld"}}