# What I learned building pipeline-aware content variants in a static Astro directory

> Source: <https://dev.to/morinaga/what-i-learned-building-pipeline-aware-content-variants-in-a-static-astro-directory-1op4>
> Published: 2026-06-13 22:13:27+00:00

**Conclusion first**: you can encode meaningful editorial differentiation into static Astro pages at build time using a single metadata field — `pipeline_tag`

from HuggingFace — without calling Claude per page. It costs nothing extra at runtime. The tradeoff is a longer page component and imprecise tags for roughly 20–25% of your dataset.

When I first deployed the AI tools directory at aiappdex.com, each model's detail page used the same structure: a Claude-generated summary, use cases, pros, cons, and a generic Amazon sidebar. The summaries were legitimately different — that's what [batch ETL with the shared Claude Haiku client](https://dev.to/articles/shared-claude-haiku-client-prompt-caching) buys you. But everything below the fold was structurally identical.

That's fine for a zero-traffic launch. It stops being fine once you start thinking about whether a user landing on a Whisper page actually gets useful guidance. The pro "Open weights available" and con "Requires evaluation for production use" mean nothing specific to audio processing. They read like database filler because they are database filler — a fallback that my [three-tier content quality ladder](https://dev.to/articles/three-tier-content-quality-ladder-programmatic-etl) generates when Claude hits a rate limit or returns unparseable JSON.

I didn't want to call Claude for every page view. That breaks static generation entirely. I also didn't want more batch jobs during ETL for every category of guidance I might want to add later. I had a simpler tool already in the data: `pipeline_tag`

.

On HuggingFace, every model has an optional `pipeline_tag`

field — a short string like `text-generation`

, `sentence-similarity`

, `automatic-speech-recognition`

, `image-classification`

, or `text-to-image`

. The [full list of supported pipeline tags is documented in the HuggingFace Hub docs](https://huggingface.co/docs/hub/models-widgets#the-pipeline-tag). It's not validated by humans in most cases; it's whatever the author put in the model card. That means it's often accurate, sometimes missing, and occasionally wrong.

My ETL stores `pipeline_tag`

in [Turso libSQL](https://dev.to/articles/turso-libsql-vs-cloudflare-d1-astro-monorepo) during the fetch step:

```
// fetch-models.ts
await db.execute({
  sql: `INSERT INTO models (id, pipeline_tag, ...)
        VALUES (?, ?, ...)
        ON CONFLICT(id) DO UPDATE SET
          pipeline_tag = excluded.pipeline_tag`,
  args: [m.modelId, m.pipeline_tag ?? null, ...],
});
```

At build time, Astro's `getStaticPaths`

loads the exported `models.json`

. So I have `pipeline_tag`

available in every `[slug].astro`

render with no API calls at all.

The core of the approach is what I'm calling `decisionPaths`

— an array of `{ when, what }`

objects rendered as a "When to use / When not to" section. Instead of generic guidance, each path is tailored to the actual model type:

``` js
const isLLM = /text-generation|conversational|chat/i.test(model.pipeline_tag ?? "");
const isEmbedding = /sentence-similarity|feature-extraction/i.test(model.pipeline_tag ?? "");
const isVision = /image|vision|object-detection|depth-estimation|segmentation/i.test(model.pipeline_tag ?? "");
const isAudio = /audio|speech|whisper|tts/i.test(model.pipeline_tag ?? "");
const isClassification = /classification/i.test(model.pipeline_tag ?? "");

const decisionPaths: { when: string; what: string }[] = [];

if (isLLM) {
  decisionPaths.push({
    when: "You need a chat-style assistant that runs on your own hardware",
    what: `${model.name} is one option, but compare quantization-friendly variants — int4 GGUF builds typically lose fewer than 2 points on benchmarks while halving VRAM.`,
  });
  decisionPaths.push({
    when: "You're prototyping and need fastest time-to-token",
    what: `Don't self-host yet — call a hosted endpoint, validate your prompts, then move to ${model.name} only when latency or unit economics force the migration.`,
  });
}
if (isEmbedding) {
  decisionPaths.push({
    when: "You're building semantic search over fewer than 1M chunks",
    what: `Check the dimension count in the tags sidebar. For small corpora, prefer 384-dim models; larger dimensions cost more in vector storage without meaningful recall improvement at that scale.`,
  });
}
```

I also added tiered commentary on the download count:

``` js
const downloadsTier =
  model.downloads >= 10_000_000 ? "established"
  : model.downloads >= 100_000 ? "actively-used"
  : model.downloads >= 1_000 ? "niche"
  : "obscure";

const downloadsCommentary =
  downloadsTier === "established"
    ? `${model.downloads.toLocaleString()} downloads — you'll find StackOverflow answers and Colab notebooks for almost any error message.`
    : downloadsTier === "niche"
    ? `${model.downloads.toLocaleString()} downloads — budget time for reading the original paper or repo issues; community knowledge is thin.`
    : ...;
```

This transforms a raw number into a usability signal without any runtime cost.

A third branch controls which affiliate sidebar appears. LLM and vision model pages show RunPod and Vast.ai GPU rental links — contextually relevant, because those users are likely to self-host and need GPU access. Embedding and classification pages don't show GPU affiliates; they get a different sidebar. The affiliate link helpers come from the [shared monetization package I wrote after abandoning AdSense](https://dev.to/articles/why-affiliate-beats-adsense-new-ai-directories).

**pipeline_tag is imprecise.** The regex `text-generation`

catches both 70B parameter LLMs and 50M classifier-style models that output text. I've had models tagged `text-generation`

that were clearly sentence transformers, misclassified by their author. The wrong guidance is worse than generic guidance — a user following LLM-tuned advice to look for GGUF quantizations for what is actually an embedding model will waste time.

**Null tags are common.** About 20–25% of models in my dataset have `pipeline_tag: null`

. For those, I fall back to a single generic decision path:

```
if (decisionPaths.length === 0) {
  decisionPaths.push({
    when: "You need a self-hosted open-source model",
    what: `Read ${model.name}'s model card on HuggingFace — pipeline category isn't clear from the metadata alone. Evaluate on your target task before committing.`,
  });
}
```

Honest, but weak. My plan for the next ETL iteration is to infer pipeline from `library_name`

— the field is more reliable for some families (`diffusers`

strongly implies vision/image generation, `sentence-transformers`

implies embedding) even when `pipeline_tag`

is null or wrong.

**The conditionals multiply fast.** Six pipeline branches, four download tiers, two affiliate blocks, one [noindex gate for template content](https://dev.to/articles/noindex-gate-programmatic-pages-without-404s) — the page component runs to about 180 lines of frontmatter script. It doesn't feel unmaintainable yet. At 300 lines I'd refactor into a `buildPageContext(model: ModelEntry)`

helper.

Worth mentioning alongside pipeline branching: I use the quality of generated content to decide whether a page gets indexed at all.

Template content — the fallback that runs when Claude fails — always has the same `pros`

array:

``` js
const TEMPLATE_PROS = JSON.stringify(["Open weights available", "Community support on HuggingFace"]);
const isTemplateContent = JSON.stringify(model.pros) === TEMPLATE_PROS;
const noindex = isTemplateContent;
```

Pages where `noindex`

is true get `<meta name="robots" content="noindex">`

injected at build. The page still builds and serves (useful for human review), but it doesn't enter the index until the ETL runs again and upgrades the content. I wrote about this pattern in detail in [How I kept programmatic pages alive while hiding them from Google](https://dev.to/articles/noindex-gate-programmatic-pages-without-404s).

The pipeline branching and the noindex gate compose naturally: a model with `null`

pipeline_tag that also has template content gets weak decision paths *and* a noindex flag, which is exactly the right editorial stance for an obscure model nobody asked for.

Build time is flat — adding six conditionals per page in Astro's SSG adds negligible overhead. 400 pages build in under 90 seconds on Vercel. Lighthouse scores didn't change because the branching generates HTML, not JavaScript.

What I can't tell you yet is whether pipeline-aware pages actually drive better [E-E-A-T signals](https://dev.to/articles/eeat-transparency-pages-programmatic-directory) or lower bounce rates than the generic fallback pages. The sites are four weeks old. I'll post Search Console data at month two; if pipeline-aware pages show meaningfully higher average engagement than null-tag fallback pages, that'll confirm the work was worth the extra complexity.

**Tag inference from library_name first.** `library_name`

is a more reliable signal for some pipeline families than `pipeline_tag`

. I'd build `inferPipeline(model)`

that tries `pipeline_tag`

first, falls back to `library_name`

mapping, then falls back to tag pattern matching.

**Config object over raw regex.** Right now each `is*`

boolean is a one-liner regex. A `PIPELINE_PROFILES`

map would be testable:

``` js
const PIPELINE_PROFILES = {
  llm: { pattern: /text-generation|conversational/i, paths: [...] },
  embedding: { pattern: /sentence-similarity|feature-extraction/i, paths: [...] },
} satisfies Record<string, { pattern: RegExp; paths: DecisionPath[] }>;
```

**Move noindex logic to lib/models.ts.** Right now `isTemplateContent`

is computed in the frontmatter script. It belongs in a utility function I can unit test without rendering a page.

**Ship the generic version first.** I spent three hours on this in week two. In retrospect I'd have launched with generic pages and added branching only after Search Console showed which pipeline categories were getting impressions. Premature differentiation on pages that aren't indexed yet adds code risk with no measurable return.

**Related articles:**

**Why not call Claude for every page view?**

Runtime AI calls make Astro's static generation impossible and add latency and API cost to every user. The [shared Claude Haiku client](https://dev.to/articles/shared-claude-haiku-client-prompt-caching) runs in a scheduled GitHub Actions job against the database, not against individual page renders.

**Doesn't the branching add a lot of maintenance surface?**

Yes — that's the honest tradeoff. The page component is about 180 lines of frontmatter script today. The alternative would be a CMS with custom fields per pipeline type, which is a larger operational burden for a solo developer. I'd revisit this call if the project had two or more people working on it.

**What if pipeline_tag is wrong for a specific model?**

The user sees guidance calibrated to the wrong pipeline type. In my spot check of about 50 models, 45 had accurate or absent tags; only 5 were actively misleading. I accept that error rate at this stage. A feedback loop — a "report incorrect info" link on each page — is on the roadmap.

**Does Google see different content per page?**

Yes. Google crawls each URL independently and sees the final rendered HTML. The Whisper model page and a LLaMA page have structurally different content sections — which is what the [E-E-A-T transparency work](https://dev.to/articles/eeat-transparency-pages-programmatic-directory) is trying to reinforce.

**Is this worth building before you have any traffic?**

Probably not. I'd ship generic pages first, measure which pipeline categories get organic impressions, then add differentiation only where it matters. Building pipeline branching before launch optimizes something you can't yet measure.

*Part of an ongoing 6-month experiment running three AI-curated directory sites. The technical claims here are real; this article was AI-assisted.*