{"slug": "how-i-built-ai-model-comparison-pages-pairwise-grouping-claude-content-libsql", "title": "How I built AI model comparison pages: pairwise grouping, Claude content, libSQL dedup", "summary": "A developer built a pipeline to auto-generate AI model comparison pages for aiappdex.com, pairing models by HuggingFace pipeline tag and using Claude Haiku to produce structured side-by-side evaluations. The approach groups the top 4 models per task category to keep comparisons relevant while capping total pairs at 50 per run for cost efficiency.", "body_md": "My AI tools directory ([aiappdex.com](https://aiappdex.com)) launched with individual model pages — one page per HuggingFace model, with download counts, tags, and Claude-generated pros and cons. That's a minimum viable directory. But directory users don't only search for a model by name; they search for two models side by side. \"Llama 3 vs Mistral 7B for a local inference project.\" \"Which text-classification checkpoint fits my fine-tuning budget.\" Comparison pages serve a different intent than detail pages, and I built a pipeline to generate them at scale.\n\nThis is how the pipeline works, what its failure modes are, and what I'd change if I were starting over.\n\nThe argument for comparison pages is search intent. A user who types \"[model A] vs [model B]\" is further along in an evaluation than one who types \"[model A].\" They already know which models they're considering; they want a structured side-by-side. That's a more specific intent signal.\n\nThe argument against auto-generating them is quality. If I ask Claude to compare two models it knows nothing concrete about and produce a paragraph of accurate-sounding generalities, I'm not adding value — I'm producing plausible boilerplate. The prompting had to produce content specific enough to be useful, or at least honest enough not to actively mislead.\n\n[The experiment I'm running](https://dev.to/morinaga/i-built-3-programmatic-seo-sites-for-25month-using-claude-haiku-heres-the-full-architecture-3pl8) has a limited monthly budget (~$25), so the comparison generation had to be economical. I'm using Claude Haiku for this step, for the same reasons I use it across all three directory ETLs: fast, cheap, and reliably produces well-formed JSON when the prompt is constrained.\n\nWith ~1,500 models in the database, the number of possible pairs is roughly 1.1 million. Generating and caching a million comparisons isn't feasible. I needed to reduce that to something tractable and coherent.\n\n**First instinct: pair by download count.** The top-N models globally give you the highest-profile pairs — Llama vs Mistral, bert-base-uncased vs roberta-base. This produces a handful of defensible comparisons for popular queries, but it ignores the long tail almost entirely. My directory is specifically built for the long tail where the big-name aggregators don't focus.\n\n**What I settled on: group by pipeline tag, then pair the top 4 within each group.** HuggingFace's pipeline tags roughly correspond to task categories: `text-generation`\n\n, `text-classification`\n\n, `image-to-image`\n\n, `token-classification`\n\n, and so on. A user evaluating a text-classification model cares about comparisons to other text-classification models, not about image generators from the same organization.\n\n``` js\nconst byPipe = new Map<string, typeof models>();\nfor (const m of models) {\n  if (!m.pipeline_tag) continue;\n  const arr = byPipe.get(m.pipeline_tag) ?? [];\n  arr.push(m);\n  byPipe.set(m.pipeline_tag, arr);\n}\n\nconst pairs: Array<[Model, Model]> = [];\nfor (const [, list] of byPipe) {\n  const sorted = [...list].sort((a, b) => b.downloads - a.downloads);\n  const take = sorted.slice(0, Math.min(4, sorted.length));\n  for (let i = 0; i < take.length; i++) {\n    for (let j = i + 1; j < take.length; j++) {\n      pairs.push([take[i]!, take[j]!]);\n    }\n  }\n}\n```\n\nFor a pipeline tag with 4 top models, that produces 6 pairs (4 choose 2). Across roughly 30 active pipeline tags in the database, that's around 180 pairs total — manageable, and where each pair is at least plausibly relevant to the same user.\n\nI cap total pairs per run at 50 using a `COMPARE_LIMIT`\n\nenv variable. Most runs skip most of those pairs because they already exist in the database.\n\nThe generation step asks Haiku for a structured JSON comparison. My general approach to Claude JSON extraction carries directly into this pipeline: constrain the system prompt to a single JSON schema, match-and-parse the response, fall back to a template on failure.\n\nThe system prompt:\n\n```\nYou compare two AI models for a technical directory. Given two model names and their metadata, produce a structured comparison. Output ONLY a JSON object:\n{ \"summary\": \"2-sentence overview\", \"differences\": [\"3-5 bullet points on key differences\"], \"similarities\": [\"2-3 bullet points on what they share\"], \"recommendation\": \"1-2 sentence guidance on which to pick in which situation\" }\nBe concrete and developer-focused. No markdown, no prose outside the JSON.\n```\n\nThe user prompt passes both models' names, authors, pipeline tags, and their existing summary strings from the database (capped at 400 characters each). That gives Haiku concrete inputs without requiring it to hallucinate architecture details.\n\nThe fallback template is not good content, but it's not wrong content either:\n\n``` js\nconst fb: CompareData = {\n  summary: `${a.name} and ${b.name} are both ${a.pipeline_tag} models. See each entry for specifics.`,\n  differences: [\"See individual model pages for architecture and use cases.\"],\n  similarities: [\"Both are open-source models on HuggingFace.\"],\n  recommendation: \"Pick based on your compute budget and specific task requirements.\",\n};\n```\n\nThis template is honest about what it doesn't know. The page renders, the structure is valid, and if I run the pipeline again with a working API key, the caching layer (described next) handles replacement cleanly.\n\nThe most important design decision in this pipeline is the dedup layer. Every pair gets a deterministic slug:\n\n``` js\nconst pairSlug = [a.slug, b.slug].sort().join(\"--vs--\");\n```\n\nSorting before joining ensures `model-a--vs--model-b`\n\nand `model-b--vs--model-a`\n\nproduce the same key regardless of which model appears first in the loop iteration. This slug is a `UNIQUE`\n\ncolumn in the `model_compare`\n\ntable.\n\nBefore calling Claude for any pair, the pipeline checks:\n\n``` js\nconst existing = await db.execute({\n  sql: `SELECT 1 FROM model_compare WHERE pair_slug = ?`,\n  args: [pairSlug]\n});\nif (existing.rows.length > 0) { skipped++; continue; }\n```\n\nExisting rows are never regenerated. This makes the daily cron idempotent with no extra bookkeeping: the table itself is the state. A run that processes 50 candidates might generate only 3-5 new comparisons if most pairs are already cached.\n\n[Turso (libSQL) is my database layer for all three sites](https://dev.to/morinaga/turso-libsql-vs-cloudflare-d1-for-an-astro-monorepo-the-practical-difference-3c9). The pair_slug UNIQUE constraint relies on slug stability — [the slug collision I fixed last week](https://dev.to/morinaga/how-i-fixed-the-slug-collision-that-silenced-aiappdexcom-for-36-hours-56e7) is relevant here because a pair_slug composed of two model slugs inherits any instability from either component slug. If model A's slug changes between runs, the old pair row becomes an orphan.\n\nAfter all pairs are processed, the pipeline dumps the full `model_compare`\n\ntable to a JSON file:\n\n``` js\nconst all = await db.execute(`SELECT * FROM model_compare ORDER BY slug_a, slug_b`);\nconst entries = all.rows.map((r) => ({\n  slug_a: String(r.slug_a),\n  slug_b: String(r.slug_b),\n  pair_slug: String(r.pair_slug),\n  summary: r.summary ? String(r.summary) : \"\",\n  differences: r.differences ? JSON.parse(String(r.differences)) : [],\n  similarities: r.similarities ? JSON.parse(String(r.similarities)) : [],\n  recommendation: r.recommendation ? String(r.recommendation) : \"\",\n}));\nawait writeFile(\"./src/data/compare.json\", JSON.stringify(entries, null, 2));\n```\n\nAstro generates a static page for each `pair_slug`\n\nat `/compare/[pair_slug]`\n\n. [I'm using SSG because static pages cost nothing to serve at current traffic levels](https://dev.to/morinaga/why-im-betting-static-ssg-beats-dynamic-ai-rendering-for-directory-seo-1pbd) and the comparison content doesn't change per-visitor.\n\nThe `differences`\n\nand `similarities`\n\narrays are stored as JSON strings in libSQL — a JSON blob approach rather than normalized rows. For a few hundred comparisons, this is fast enough, and it keeps the schema simple.\n\n**The fallback template makes the pipeline crash-proof.** Running without an API key, or with an exhausted budget, still produces structurally valid pages. The CI step doesn't fail; it just produces lower-quality content. I can re-run the pipeline later with a working key and the cache miss detection handles content upgrades automatically.\n\n**pair_slug as the dedup key is clean.** No separate \"already processed\" log file, no state outside the database. If the row exists, skip. If not, generate. This is the same pattern I use for the HuggingFace model fetch ETL — the database table is the state.\n\n**Pipeline-tag grouping produces coherent pairs.** The comparisons are topically relevant by construction. A visitor on an `image-to-image`\n\nmodel page sees comparisons to other `image-to-image`\n\nmodels, not cross-category noise.\n\n**Pure download-count ranking picks the boring pairs.** The top-downloaded models in each pipeline tag are often the most generic: `bert-base-uncased`\n\n, `gpt2`\n\n, tutorial-tier checkpoints. The interesting comparisons for a developer with a real task are often between models ranked 5th and 12th in a category — specific enough to have real architectural differences, popular enough to have community benchmarks. I'd add a recency weight (something like `recent_30d_downloads / total_downloads`\n\n) to surface models that are gaining traction rather than coasting on historical popularity.\n\n**Sequential generation is slow.** Right now pairs run one at a time. With 50 pairs per run and ~1-2 seconds per Haiku call, that's a 1-2 minute sequential run. Async batching at 5-10 concurrent calls with a rate limiter would cut that to 10-20 seconds. I haven't prioritized this because the run happens in a CI job that's off the critical path, but it would make iteration faster.\n\n**The prompt fields are too open-ended.** Asking Claude to list \"differences\" with no axis constraints produces inconsistent comparisons — one pair gets architecture differences, another gets license differences, another gets vague use-case differences. The output isn't comparable across pairs. I'd specify the axes explicitly: \"List differences specifically in: model architecture, license, parameter count, supported languages, inference cost.\"\n\n**Comparison pages aren't linked from model detail pages yet.** A visitor on the Llama 3.2 page should see \"Compare Llama 3.2 with:\" and a list of its cached comparisons. I have the data to build this; the UI just isn't wired. I'm adding it next.\n\n**How many comparison pages are there now?**\n\nI don't have an exact count to publish yet — I'll run a `SELECT COUNT(*) FROM model_compare`\n\nand post an update in 30 days. The theoretical cap from the pairing logic is around 180 active pairs.\n\n**Does Claude ever refuse to compare two models?**\n\nNot in practice. The prompt is framed as \"produce a developer-focused comparison JSON,\" which is a concrete technical task. The closest failure mode is Haiku producing generic differences when both models are obscure fine-tunes with minimal distinguishing metadata — technically valid output, not particularly useful.\n\n**What happens when a model gets deleted from HuggingFace?**\n\nThe comparison row stays in the database. The compare page stays live with stale content. I don't have a cleanup pipeline for deleted models yet. The public HuggingFace API doesn't expose a \"deleted models\" feed, so detection requires checking 404 responses during the nightly model refresh — something I'll add once I have a reason to care about page count accuracy.\n\n**Why not use embedding similarity to pick the most relevant models to compare?**\n\nBudget and complexity. Embedding 1,500 model summaries and running similarity queries needs either a vector extension (Turso has pgvector) or an external store. For a site with effectively zero traffic today, that infrastructure overhead isn't justified. The pipeline_tag grouping achieves \"plausibly relevant pairs\" with zero extra infrastructure cost, and I can upgrade the pairing logic later without changing the generation or caching layers.\n\n*Part of an ongoing 6-month experiment running three AI-curated directory sites. The technical claims here are real; this article was AI-assisted.*", "url": "https://wpnews.pro/news/how-i-built-ai-model-comparison-pages-pairwise-grouping-claude-content-libsql", "canonical_source": "https://dev.to/morinaga/how-i-built-ai-model-comparison-pages-pairwise-grouping-claude-content-libsql-dedup-3o7m", "published_at": "2026-07-29 08:16:38+00:00", "updated_at": "2026-07-29 08:36:28.300009+00:00", "lang": "en", "topics": ["artificial-intelligence", "developer-tools", "large-language-models"], "entities": ["HuggingFace", "Claude Haiku", "aiappdex.com"], "alternates": {"html": "https://wpnews.pro/news/how-i-built-ai-model-comparison-pages-pairwise-grouping-claude-content-libsql", "markdown": "https://wpnews.pro/news/how-i-built-ai-model-comparison-pages-pairwise-grouping-claude-content-libsql.md", "text": "https://wpnews.pro/news/how-i-built-ai-model-comparison-pages-pairwise-grouping-claude-content-libsql.txt", "jsonld": "https://wpnews.pro/news/how-i-built-ai-model-comparison-pages-pairwise-grouping-claude-content-libsql.jsonld"}}