{"slug": "the-llm-suggests-code-decides-building-a-cost-aware-router-for-image-and-video", "title": "The LLM Suggests; Code Decides: Building a Cost-Aware Router for Image and Video Models", "summary": "A developer detailed the architecture of a cost-aware router for image and video generation models, emphasizing that LLMs should act as planners rather than final decision-makers. The system uses deterministic intent classification, tier-filtered model catalogs, and server-side validation to ensure reliable and cost-effective model selection.", "body_md": "AI model routing looks deceptively simple:\n\nThat approach works in a demo. It becomes unreliable as soon as a product supports text-to-image, image editing, multi-reference fusion, and video—each with different aspect ratios, durations, pricing rules, and access tiers.\n\nThe production lesson was straightforward: treat the LLM as a **planner**, not as the final authority.\n\nThe final decision should come from a constrained pipeline:\n\n```\nuser brief\n   ↓\ndeterministic intent signals\n   ↓\ntask kind + input capability\n   ↓\ntier-filtered model catalog\n   ↓\nLLM proposal\n   ↓\nserver-side capability and cost validation\n   ↓\nexecutable generation plan\n```\n\nThis article explains that architecture, the failure modes behind it, and the invariants worth testing.\n\nThe first mistake is jumping directly from a prompt to a model.\n\nBefore selecting a model, identify the actual generation task:\n\n```\ntype GenerationKind =\n  | \"image-t2i\"\n  | \"image-i2i\"\n  | \"image-fusion\"\n  | \"video\";\n\ntype AgentMode = \"create\" | \"edit\" | \"fusion\";\n\nfunction modelKindFor(\n  mediaType: \"image\" | \"video\",\n  referenceCount: number,\n  mode: AgentMode\n): GenerationKind {\n  if (mediaType === \"video\") return \"video\";\n  if (mode === \"fusion\" || referenceCount >= 2) return \"image-fusion\";\n  if (mode === \"edit\" || referenceCount === 1) return \"image-i2i\";\n  return \"image-t2i\";\n}\n```\n\nReference count is not merely metadata.\n\nThis classification dramatically reduces the number of models the planner must consider.\n\nA pure keyword router is too brittle. A pure LLM router is too nondeterministic.\n\nA hybrid router works better:\n\n``` js\nfunction classifyMediaType(text: string): \"image\" | \"video\" | null {\n  const normalized = text.toLowerCase();\n\n  if (IMAGE_TO_VIDEO_PATTERNS.some((pattern) => pattern.test(normalized))) {\n    return \"video\";\n  }\n\n  if (IMAGE_SIGNALS.some((signal) => normalized.includes(signal))) {\n    return \"image\";\n  }\n\n  if (VIDEO_SIGNALS.some((signal) => normalized.includes(signal))) {\n    return \"video\";\n  }\n\n  return null;\n}\n```\n\nThe order matters. Compare these briefs:\n\n```\nCreate a cinematic 3x3 photo grid\nAnimate this photo into a short cinematic video\n```\n\nBoth contain image and cinematic language. The first must remain an image task, while the second is explicitly image-to-video.\n\nStrong signals should be handled deterministically; only genuinely ambiguous requests need an LLM classifier. Recording whether a decision came from `rules`\n\n, `llm`\n\n, or `fallback`\n\nalso makes misroutes much easier to debug.\n\nThe planner should not rely on its training data to remember model constraints.\n\nKeep a catalog that code can validate:\n\n```\ninterface ModelSpec {\n  id: string;\n  displayName: string;\n  kinds: GenerationKind[];\n  minimumTier: \"free\" | \"premium\" | \"ultimate\";\n  baseCredits: number;\n  resolutions: string[];\n  aspectRatios: string[];\n  formats: Array<\"jpg\" | \"png\" | \"webp\" | \"mp4\">;\n  tags: string[];\n}\n```\n\nThe LLM receives only the models available for the current task and membership tier:\n\n```\nfunction catalogForTier(\n  tier: Tier,\n  kind: GenerationKind\n): ModelSpec[] {\n  return catalog.filter(\n    (model) =>\n      model.kinds.includes(kind) &&\n      tierRank(model.minimumTier) <= tierRank(tier)\n  );\n}\n```\n\nThis is cheaper and safer than giving the planner a long list of unusable models.\n\nOne practical detail: browser components often need capability metadata but must not import provider SDKs. Keep a client-safe projection with no server clients or environment-dependent imports. Ideally, generate that projection from the authoritative registry and test it for drift.\n\nModel selection is connected to prompt enhancement and output parameters, so the planner returns one structured object:\n\n```\ninterface GenerationPlan {\n  intent: string;\n  enhancedPrompt: string;\n  recommendedModel: string;\n  aspectRatio: string;\n  resolution: string;\n  outputFormat: \"jpg\" | \"png\" | \"webp\" | \"mp4\";\n  estimatedCredits: number;\n  modelReason: string;\n  warnings?: string[];\n  videoPlan?: {\n    shots: Array<{\n      order: number;\n      description: string;\n      duration: number;\n      camera?: string;\n    }>;\n    totalDuration: number;\n  };\n}\n```\n\nRequest JSON output, but assume it can still fail. Production systems eventually see:\n\nThe response is a proposal. It is not executable until the server validates it.\n\nThe most important part of the router happens after planning.\n\nStart by canonicalizing aliases, then enforce task kind and tier:\n\n``` js\nconst canonicalId = canonicalModelId(plan.recommendedModel);\nconst model = coerceModel(canonicalId, userTier, generationKind);\n```\n\nIf a free user is assigned a premium-only model, downgrade to a safe model and emit a warning. If the ID is unknown, use the default for that tier and task instead of sending a broken provider request.\n\nThen add a small number of high-confidence capability vetoes.\n\nFor image-to-image work, these requests are not equivalent:\n\n```\nRemove the person on the left\nReimagine this portrait as an editorial photoshoot\n```\n\nThe first is a surgical local edit. The second needs composition changes and identity preservation. A model optimized for object removal may perform poorly on the second request even though both are technically image-to-image.\n\nThe LLM can still make the normal recommendation. Deterministic code should override it only when the mismatch is clear. Rules should veto impossible or predictably bad choices, not replace every subjective creative decision.\n\nA router becomes frustrating if it ignores explicit choices.\n\nFor output parameters, use a clear priority order:\n\n``` js\nconst requestedAspect =\n  input.pinnedAspect ??\n  extractAspectFromBrief(input.brief) ??\n  recommendAspect(input, model.aspectRatios) ??\n  plan.aspectRatio ??\n  model.aspectRatios[0];\n\nconst finalAspect = model.aspectRatios.includes(requestedAspect)\n  ? requestedAspect\n  : closestSupportedAspect(requestedAspect, model.aspectRatios);\n```\n\nApply the same process to resolution, output format, and video duration.\n\nDuration deserves special handling because providers expose very different contracts. One model may allow any duration from four to fifteen seconds, while another accepts only four, six, or eight seconds. Never let a natural-language explanation promise ten seconds if the actual provider call will use eight.\n\nDo not trust a cost estimate generated before the final model, resolution, duration, reference count, and output count are known.\n\nThe reliable order is:\n\n```\nchoose model\n→ validate model\n→ resolve final parameters\n→ calculate final cost\n→ show confirmation\n→ execute\njs\nconst credits = calculateCredits({\n  modelId: finalModel.id,\n  resolution: finalResolution,\n  duration: finalDuration,\n  referenceCount,\n  outputCount\n});\n```\n\nThe confirmation card and billing service must use the same pricing function. Otherwise, an agent can recommend a 40-credit plan that deducts 80 credits after generation.\n\nCost-aware routing also needs a simple principle: when two models satisfy the same hard requirements, prefer the cheaper one. Step up only when the brief asks for a capability the cheaper model lacks.\n\nThe planner will eventually time out, return invalid JSON, or become unavailable.\n\nA generation product should still return a valid plan. A useful fallback combines:\n\nThe fallback is less nuanced than an LLM-generated plan, but it remains executable and honest.\n\nRetries also need idempotency. A timed-out request should return the previous result, not create a second expensive generation. We use request idempotency keys and stored receipts so that replaying the same step is safe.\n\nSnapshotting an LLM explanation provides little confidence. Test invariants instead:\n\n```\nassert(classifyMediaType(\"cinematic 3x3 photo grid\") === \"image\");\n\nassert(\n  classifyMediaType(\"turn this photo into a 6-second video\") === \"video\"\n);\n\nassert(\n  coerceModel(\"ultimate-only-model\", \"free\", \"image-t2i\").downgraded\n);\n\nassert(\n  classifyImageEditIntent(\"remove the watermark\") === \"precise-edit\"\n);\n\nassert(\n  classifyImageEditIntent(\"editorial reinterpretation\") === \"style-transfer\"\n);\n```\n\nOther valuable regression cases include:\n\nMany of the most valuable routing tests begin as real user-visible mistakes.\n\nThe biggest improvement was not smarter prose. It was predictability.\n\nThe agent could explain:\n\nThe general lesson is simple:\n\nUse the LLM to understand creative intent. Use deterministic code to enforce reality.\n\nThat separation makes it easier to add models, change pricing, retire providers, and reproduce routing decisions without rewriting one giant prompt.\n\nThe production UI that exercises this router is Ava. You can inspect the plan, selected model, rationale, and quoted cost in the working implementation: [see the routing flow](https://createvision.ai/agent).\n\nWhat routing rule would you keep deterministic even if the planner improves?\n\n*Disclosure: I used AI to help structure and edit this draft. I reviewed the production code, examples, and technical claims before publication.*", "url": "https://wpnews.pro/news/the-llm-suggests-code-decides-building-a-cost-aware-router-for-image-and-video", "canonical_source": "https://dev.to/yestwind_3e558e544559c15a/the-llm-suggests-code-decides-building-a-cost-aware-router-for-image-and-video-models-297", "published_at": "2026-08-31 16:25:43+00:00", "updated_at": "2026-08-31 16:52:30.256792+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "generative-ai", "ai-products", "developer-tools"], "entities": [], "alternates": {"html": "https://wpnews.pro/news/the-llm-suggests-code-decides-building-a-cost-aware-router-for-image-and-video", "markdown": "https://wpnews.pro/news/the-llm-suggests-code-decides-building-a-cost-aware-router-for-image-and-video.md", "text": "https://wpnews.pro/news/the-llm-suggests-code-decides-building-a-cost-aware-router-for-image-and-video.txt", "jsonld": "https://wpnews.pro/news/the-llm-suggests-code-decides-building-a-cost-aware-router-for-image-and-video.jsonld"}}