How I built AI model comparison pages: pairwise grouping, Claude content, libSQL dedup

A developer built a pipeline to auto-generate AI model comparison pages for aiappdex.com, pairing models by HuggingFace pipeline tag and using Claude Haiku to produce structured side-by-side evaluations. The approach groups the top 4 models per task category to keep comparisons relevant while capping total pairs at 50 per run for cost efficiency.

My AI tools directory aiappdex.com https://aiappdex.com launched with individual model pages — one page per HuggingFace model, with download counts, tags, and Claude-generated pros and cons. That's a minimum viable directory. But directory users don't only search for a model by name; they search for two models side by side. "Llama 3 vs Mistral 7B for a local inference project." "Which text-classification checkpoint fits my fine-tuning budget." Comparison pages serve a different intent than detail pages, and I built a pipeline to generate them at scale. This is how the pipeline works, what its failure modes are, and what I'd change if I were starting over. The argument for comparison pages is search intent. A user who types " model A vs model B " is further along in an evaluation than one who types " model A ." They already know which models they're considering; they want a structured side-by-side. That's a more specific intent signal. The argument against auto-generating them is quality. If I ask Claude to compare two models it knows nothing concrete about and produce a paragraph of accurate-sounding generalities, I'm not adding value — I'm producing plausible boilerplate. The prompting had to produce content specific enough to be useful, or at least honest enough not to actively mislead. The experiment I'm running https://dev.to/morinaga/i-built-3-programmatic-seo-sites-for-25month-using-claude-haiku-heres-the-full-architecture-3pl8 has a limited monthly budget ~$25 , so the comparison generation had to be economical. I'm using Claude Haiku for this step, for the same reasons I use it across all three directory ETLs: fast, cheap, and reliably produces well-formed JSON when the prompt is constrained. With ~1,500 models in the database, the number of possible pairs is roughly 1.1 million. Generating and caching a million comparisons isn't feasible. I needed to reduce that to something tractable and coherent. First instinct: pair by download count. The top-N models globally give you the highest-profile pairs — Llama vs Mistral, bert-base-uncased vs roberta-base. This produces a handful of defensible comparisons for popular queries, but it ignores the long tail almost entirely. My directory is specifically built for the long tail where the big-name aggregators don't focus. What I settled on: group by pipeline tag, then pair the top 4 within each group. HuggingFace's pipeline tags roughly correspond to task categories: text-generation , text-classification , image-to-image , token-classification , and so on. A user evaluating a text-classification model cares about comparisons to other text-classification models, not about image generators from the same organization. js const byPipe = new Map<string, typeof models ; for const m of models { if m.pipeline tag continue; const arr = byPipe.get m.pipeline tag ?? ; arr.push m ; byPipe.set m.pipeline tag, arr ; } const pairs: Array< Model, Model = ; for const , list of byPipe { const sorted = ...list .sort a, b = b.downloads - a.downloads ; const take = sorted.slice 0, Math.min 4, sorted.length ; for let i = 0; i < take.length; i++ { for let j = i + 1; j < take.length; j++ { pairs.push take i , take j ; } } } For a pipeline tag with 4 top models, that produces 6 pairs 4 choose 2 . Across roughly 30 active pipeline tags in the database, that's around 180 pairs total — manageable, and where each pair is at least plausibly relevant to the same user. I cap total pairs per run at 50 using a COMPARE LIMIT env variable. Most runs skip most of those pairs because they already exist in the database. The generation step asks Haiku for a structured JSON comparison. My general approach to Claude JSON extraction carries directly into this pipeline: constrain the system prompt to a single JSON schema, match-and-parse the response, fall back to a template on failure. The system prompt: You compare two AI models for a technical directory. Given two model names and their metadata, produce a structured comparison. Output ONLY a JSON object: { "summary": "2-sentence overview", "differences": "3-5 bullet points on key differences" , "similarities": "2-3 bullet points on what they share" , "recommendation": "1-2 sentence guidance on which to pick in which situation" } Be concrete and developer-focused. No markdown, no prose outside the JSON. The user prompt passes both models' names, authors, pipeline tags, and their existing summary strings from the database capped at 400 characters each . That gives Haiku concrete inputs without requiring it to hallucinate architecture details. The fallback template is not good content, but it's not wrong content either: js const fb: CompareData = { summary: ${a.name} and ${b.name} are both ${a.pipeline tag} models. See each entry for specifics. , differences: "See individual model pages for architecture and use cases." , similarities: "Both are open-source models on HuggingFace." , recommendation: "Pick based on your compute budget and specific task requirements.", }; This template is honest about what it doesn't know. The page renders, the structure is valid, and if I run the pipeline again with a working API key, the caching layer described next handles replacement cleanly. The most important design decision in this pipeline is the dedup layer. Every pair gets a deterministic slug: js const pairSlug = a.slug, b.slug .sort .join "--vs--" ; Sorting before joining ensures model-a--vs--model-b and model-b--vs--model-a produce the same key regardless of which model appears first in the loop iteration. This slug is a UNIQUE column in the model compare table. Before calling Claude for any pair, the pipeline checks: js const existing = await db.execute { sql: SELECT 1 FROM model compare WHERE pair slug = ? , args: pairSlug } ; if existing.rows.length 0 { skipped++; continue; } Existing rows are never regenerated. This makes the daily cron idempotent with no extra bookkeeping: the table itself is the state. A run that processes 50 candidates might generate only 3-5 new comparisons if most pairs are already cached. Turso libSQL is my database layer for all three sites https://dev.to/morinaga/turso-libsql-vs-cloudflare-d1-for-an-astro-monorepo-the-practical-difference-3c9 . The pair slug UNIQUE constraint relies on slug stability — the slug collision I fixed last week https://dev.to/morinaga/how-i-fixed-the-slug-collision-that-silenced-aiappdexcom-for-36-hours-56e7 is relevant here because a pair slug composed of two model slugs inherits any instability from either component slug. If model A's slug changes between runs, the old pair row becomes an orphan. After all pairs are processed, the pipeline dumps the full model compare table to a JSON file: js const all = await db.execute SELECT FROM model compare ORDER BY slug a, slug b ; const entries = all.rows.map r = { slug a: String r.slug a , slug b: String r.slug b , pair slug: String r.pair slug , summary: r.summary ? String r.summary : "", differences: r.differences ? JSON.parse String r.differences : , similarities: r.similarities ? JSON.parse String r.similarities : , recommendation: r.recommendation ? String r.recommendation : "", } ; await writeFile "./src/data/compare.json", JSON.stringify entries, null, 2 ; Astro generates a static page for each pair slug at /compare/ pair slug . I'm using SSG because static pages cost nothing to serve at current traffic levels https://dev.to/morinaga/why-im-betting-static-ssg-beats-dynamic-ai-rendering-for-directory-seo-1pbd and the comparison content doesn't change per-visitor. The differences and similarities arrays are stored as JSON strings in libSQL — a JSON blob approach rather than normalized rows. For a few hundred comparisons, this is fast enough, and it keeps the schema simple. The fallback template makes the pipeline crash-proof. Running without an API key, or with an exhausted budget, still produces structurally valid pages. The CI step doesn't fail; it just produces lower-quality content. I can re-run the pipeline later with a working key and the cache miss detection handles content upgrades automatically. pair slug as the dedup key is clean. No separate "already processed" log file, no state outside the database. If the row exists, skip. If not, generate. This is the same pattern I use for the HuggingFace model fetch ETL — the database table is the state. Pipeline-tag grouping produces coherent pairs. The comparisons are topically relevant by construction. A visitor on an image-to-image model page sees comparisons to other image-to-image models, not cross-category noise. Pure download-count ranking picks the boring pairs. The top-downloaded models in each pipeline tag are often the most generic: bert-base-uncased , gpt2 , tutorial-tier checkpoints. The interesting comparisons for a developer with a real task are often between models ranked 5th and 12th in a category — specific enough to have real architectural differences, popular enough to have community benchmarks. I'd add a recency weight something like recent 30d downloads / total downloads to surface models that are gaining traction rather than coasting on historical popularity. Sequential generation is slow. Right now pairs run one at a time. With 50 pairs per run and ~1-2 seconds per Haiku call, that's a 1-2 minute sequential run. Async batching at 5-10 concurrent calls with a rate limiter would cut that to 10-20 seconds. I haven't prioritized this because the run happens in a CI job that's off the critical path, but it would make iteration faster. The prompt fields are too open-ended. Asking Claude to list "differences" with no axis constraints produces inconsistent comparisons — one pair gets architecture differences, another gets license differences, another gets vague use-case differences. The output isn't comparable across pairs. I'd specify the axes explicitly: "List differences specifically in: model architecture, license, parameter count, supported languages, inference cost." Comparison pages aren't linked from model detail pages yet. A visitor on the Llama 3.2 page should see "Compare Llama 3.2 with:" and a list of its cached comparisons. I have the data to build this; the UI just isn't wired. I'm adding it next. How many comparison pages are there now? I don't have an exact count to publish yet — I'll run a SELECT COUNT FROM model compare and post an update in 30 days. The theoretical cap from the pairing logic is around 180 active pairs. Does Claude ever refuse to compare two models? Not in practice. The prompt is framed as "produce a developer-focused comparison JSON," which is a concrete technical task. The closest failure mode is Haiku producing generic differences when both models are obscure fine-tunes with minimal distinguishing metadata — technically valid output, not particularly useful. What happens when a model gets deleted from HuggingFace? The comparison row stays in the database. The compare page stays live with stale content. I don't have a cleanup pipeline for deleted models yet. The public HuggingFace API doesn't expose a "deleted models" feed, so detection requires checking 404 responses during the nightly model refresh — something I'll add once I have a reason to care about page count accuracy. Why not use embedding similarity to pick the most relevant models to compare? Budget and complexity. Embedding 1,500 model summaries and running similarity queries needs either a vector extension Turso has pgvector or an external store. For a site with effectively zero traffic today, that infrastructure overhead isn't justified. The pipeline tag grouping achieves "plausibly relevant pairs" with zero extra infrastructure cost, and I can upgrade the pairing logic later without changing the generation or caching layers. Part of an ongoing 6-month experiment running three AI-curated directory sites. The technical claims here are real; this article was AI-assisted.