# I counted my 4,323 listings as companies. The biggest one is GitHub Pages.

> Source: <https://dev.to/li_yi_dcec96bf2647dec1b7e/i-counted-my-4323-listings-as-companies-the-biggest-one-is-github-pages-136m>
> Published: 2026-11-22 00:40:38+00:00

**Key takeaway:** Collapsing websiteUrl to its registrable domain showed 1,200 of 4,323 directory listings (27.7%) share a domain with a sibling row, and 337 (7.8%) resolve to platform or affiliate hosts instead of the vendor's own site.

I run an AI tools directory. The homepage says "4,300+ tools" and the number is real: my database has exactly 4,323 rows in the `Tool` table. What I had never done is ask what one row *is*. Yesterday I found that the rank badges on those pages had drifted, so today I went one level down and measured the unit itself.

The answer: 4,323 listings, 3,317 companies, and the single largest "vendor" in my directory is a hosting platform I don't own.

One row, one name, one slug, one `websiteUrl`, one `pricingType`. The "Visit Website" button on a tool page points at `/go/{slug}`, which records a click and then 302s to whatever is in `websiteUrl`:

``` php
GET /go/descript                   -> 302 https://get.descript.com/sx9ahgvct8j9
GET /go/underlord-by-descript-ai-video -> 302 https://get.descript.com/sx9ahgvct8j9
```

Those are two different listings, two different pages, two different slugs. The second one is a feature of the first one. That is the only byte-for-byte `websiteUrl` collision in the whole table - 4,323 rows, 4,322 distinct URLs - so I initially read that as good news. It isn't. Uniqueness at the string level just means the duplicates are one level up: they point at the same *company*, just at different pages of it.

For every row I took the hostname of `websiteUrl`, dropped a leading `www.`, and then collapsed to the registrable domain with the crudest rule that works: the last two labels, with a short exception list for `co.uk` / `com.cn` style suffixes. No real public-suffix library, no brand knowledge, no verification that two domains belong to the same legal entity. So everything below is a **lower bound on how clustered my catalog is**, not a clean company count - `deepmind.google` and `blog.google` count as two domains here even though they are one company.

Nothing is unparseable: 4,323 URLs, 0 failures.

| measure | value | 
|---|---|
| rows | 4,323 | 
| distinct hostnames | 3,576 | 
| distinct registrable domains | 3,317 | 
| rows that share a domain with at least one sibling row | 1,200 (27.7%) | 
| domains involved in that sharing | 194 | 

So better than one row in four sits behind a domain that already appears elsewhere on my site. The largest clusters are not the things you'd expect from a "curated list of AI companies":

```
github.io      103     huggingface.co   78     sjv.io         28
github.com      81     openai.com       50     pxf.io         24
```

Merge `github.com` and `github.io` and GitHub is my single biggest vendor at 184 listings - more than Google (64 rows on `google.com`, plus 24 on `deepmind.google` and 19 on `blog.google`) and more than OpenAI (50). Not because GitHub has that many products in my catalog, but because a project page on someone's GitHub Pages and a repository on GitHub's own domain both get treated by my crawler as "the tool's website".

If I filter rows whose host is a platform or an affiliate network (`github.io`, `github.com`, `huggingface.co`, `sjv.io`, `pxf.io`, `prf.hn`, `partnerlinks.io`), I get 337 rows - 7.8% of the catalog - where the address I present as "official website" is not the vendor's address. Live checks, done today:

``` php
/go/adobe-firefly      -> 302 https://prf.hn/l/EJXxwOp
/go/capcut             -> 302 https://capcutaffiliateprogram.pxf.io/9LPy7W
/go/yue2-ai-model-music-> 302 https://map-yue2.github.io/
/go/trellis-2-microsoft-> 302 https://microsoft.github.io/TRELLIS.2
```

Adobe Podcast, Adobe Firefly, Looka, n8n, Klaviyo: their entry in my "official website" column is a link belonging to an affiliate network, inherited from the directory I crawl. My own affiliate store is empty - I have not signed up for a single program - so when a reader clicks those, some third party gets attributed for the visit and I get a click in my analytics and nothing else. A looser rule (network host *or* a `fp_ref=` / `ref=aixploria=` query param) catches 69 rows; 62 of them on the network's own domain.

And 303 of those 337 rows are labelled `free`, which is the filter most of my readers use. That is the part I'd fix first.

I checked the text too. Zero rows have byte-identical `description` values - but if I replace the tool's own name with a placeholder and the "ranks #N in the X category" clause with another, 4,323 descriptions collapse into 4,205 shapes. The largest shape is shared by 18 rows, all sitting in a category literally named `This*DoesNotExist` (21 tools). Their pages open with the same sentence, differing only in the product name.

The one sentence I'd delete before any of that: 2,780 rows (64.3%) end their description with "Common tags for this tool include Latest AI." "Latest AI" is not a tag - it is the name of the source site's chronological feed, and it is attached in my database to 2,860 tools, making it the most common tag I have. 1,446 rows print no tag sentence at all, and 824 have zero tag rows. So two thirds of my catalog tells the reader, in a field labelled "tags", that the item appeared recently in somebody else's list.

My content pipeline today produced 5 more written pages out of the 130 still templated in my top-150-by-traffic queue, which is the number I actually report. But the honest framing for the catalog itself is now: roughly 3,300 destinations, 4,300 listings, about 340 of them pointing at infrastructure rather than a company, and one page that is genuinely a duplicate of another page. The free filter shows 2,198 tools; behind those 2,198 rows there are 1,538 registrable domains, and 101 of them are GitHub Pages sites.

I have not deduplicated anything, and I am not going to pretend a merge is free: "Underlord by Descript" might be a listing a reader searches for on purpose. The fix I'd defend is a canonical vendor key stored next to the URL, so "same company" is a fact I can query instead of something a reader has to notice. Until then I won't write "4,323 tools" and mean 4,323 products - I mean 4,323 rows, and the [free tools list](https://aiverse.mindutil.com/free-ai-tools?utm_source=devto&utm_medium=social&utm_campaign=topic) is the page where that difference shows up most.
