How to Build a Cost-Aware AI Model Router for SaaS Workflows

A developer outlines a cost-aware AI model router design for SaaS workflows, using task profiles and model tiers to reduce inference costs. The approach replaces hard-coded model choices with task-based routing, starting with simple rules before considering machine learning.

An AI product can become expensive without doing anything obviously wrong. The prompts work. The model answers correctly. Users are getting value. Then usage grows and the inference bill grows much faster than expected. One common reason is architectural: every task is being sent through roughly the same model path. A document extraction step uses the same model as a difficult reasoning task. A simple classification gets the same reasoning effort as a complex investigation. A repeated 20,000-token workspace context gets sent again and again. A model receives hundreds of tool results just to filter and sort them. Nothing is technically broken. The workflow is simply spending expensive model intelligence on work that does not always need it. A cost-aware model router fixes that by deciding how each task should run before the request reaches the model. A weak routing design usually begins like this: js const response = await ai.generate { model: "strongest-model", input } ; Every feature eventually calls the same helper. That is easy to build. It also hides the economics. A better starting point is to describe what the task actually requires. For example: type AITask = | "extract" | "classify" | "summarize" | "support answer" | "research" | "decision" | "complex agent"; Now the application has something meaningful to route. The model choice becomes a consequence of the job instead of a hard-coded default. The task name alone is not enough. Two extraction tasks may have very different requirements. A short invoice and a 200-page legal document should not necessarily follow the same path. Represent the requirements explicitly. type TaskProfile = { task: AITask; complexity: | "low" | "medium" | "high"; latency: | "live" | "interactive" | "background"; reasoning: | "minimal" | "low" | "medium" | "high"; volume: | "low" | "medium" | "high"; deterministicPostProcessing: boolean; }; A simple extraction feature could then declare: js const profile: TaskProfile = { task: "extract", complexity: "low", latency: "interactive", reasoning: "minimal", volume: "high", deterministicPostProcessing: true }; A difficult research workflow might look very different: js const profile: TaskProfile = { task: "research", complexity: "high", latency: "background", reasoning: "high", volume: "low", deterministicPostProcessing: false }; The router now has product context. Do not start with twenty routing combinations. Three tiers are enough for many products. type ModelTier = | "economy" | "balanced" | "frontier"; Think about them by responsibility. Useful for high-volume work with predictable output. Examples: Useful when the task needs stronger interpretation but still runs frequently. Examples: Reserve this for work where better judgment materially changes the outcome. Examples: The names can change. The boundary is what matters. A first version does not need machine learning. Rules are often easier to inspect. function chooseTier profile: TaskProfile : ModelTier { if profile.complexity === "high" || profile.reasoning === "high" { return "frontier"; } if profile.complexity === "medium" || profile.reasoning === "medium" { return "balanced"; } return "economy"; } Then map each tier to the model configuration you currently prefer. js const MODEL CONFIG = { economy: { model: "cost-optimized-model", reasoningEffort: "minimal" }, balanced: { model: "balanced-model", reasoningEffort: "low" }, frontier: { model: "frontier-model", reasoningEffort: "high" } }; Keep product logic separated from provider configuration. When pricing or model performance changes, you can update the mapping without rewriting every feature. Reasoning effort is another routing decision. If the task is: Extract the invoice number, customer name, and total. high reasoning may add cost without adding useful product value. For a task like: Compare these five contracts, identify conflicting obligations, and explain which interpretation is best supported. the extra reasoning may be justified. So route reasoning independently when possible. function chooseReasoning profile: TaskProfile { if profile.reasoning === "high" { return "high"; } if profile.reasoning === "medium" { return "medium"; } return "minimal"; } This gives you another economic control without changing the user experience. This can remove surprising amounts of token usage. Imagine an agent retrieves 200 records and needs to: The model does not need to perform steps 1 through 4. That work is deterministic. Write code for it. js const relevant = records .filter isWithinLast30Days .sort a, b = b.value - a.value .slice 0, 20 ; const grouped = groupByAccount relevant ; Now send the smaller result into the model. js const judgment = await analyzeAccounts grouped ; The model spends tokens on judgment. Code handles filtering, sorting, counting, and aggregation. OpenAI highlights this same separation in its GPT-5.6 guidance, describing workflows where programmatic tool calling processes deterministic intermediate data outside the model context so model tokens stay focused on reasoning. Many SaaS AI features send large stable prefixes repeatedly. Examples include: If the first 25,000 tokens are almost identical across several requests, repeatedly processing that prefix creates unnecessary cost. Use prompt caching where the provider supports it. Also structure prompts so stable content remains stable. Bad: Timestamp Dynamic metadata Large company instructions Tool definitions User request Better: Large stable company instructions Tool definitions Stable workspace context Dynamic metadata User request The more stable the reusable prefix is, the more useful caching can become. OpenAI says GPT-5.6 extends its prompt cache TTL to at least 30 minutes and supports deterministic cache breakpoints, specifically to improve reuse across repeated agent runs. More agents do not automatically mean better architecture. Suppose the primary agent creates six subagents. Each receives context. Each calls tools. Each generates reasoning. Then another model synthesizes the outputs. That can be useful for work that genuinely benefits from parallel investigation. It can also multiply token consumption very quickly. Represent the decision explicitly. type ParallelPolicy = { allowed: boolean; maxAgents: number; minimumComplexity: "medium" | "high"; }; Then: function canSpawnSubagents profile: TaskProfile, policy: ParallelPolicy { if policy.allowed return false; if profile.complexity == "high" { return false; } return true; } The workflow should earn parallelism. Do not make subagents the default simply because the API supports them. Model routing becomes much more useful when the product has an economic boundary. Define cost at the feature level. type FeatureBudget = { feature: string; maxCostPerRunUsd: number; warningThresholdUsd: number; }; For example: js const budget: FeatureBudget = { feature: "document enrichment", maxCostPerRunUsd: 0.15, warningThresholdUsd: 0.10 }; The exact number should come from your own product economics. The architecture now knows that cost is a requirement, not merely something observed at the end of the month. Every AI run should leave enough information to explain its cost. type AIRun = { feature: string; task: AITask; selectedTier: ModelTier; reasoningEffort: string; inputTokens: number; outputTokens: number; cachedInputTokens?: number; toolCalls: number; estimatedCostUsd: number; durationMs: number; successful: boolean; }; Now you can answer useful questions. Which features are consuming the most AI budget? Which tasks regularly escalate to the frontier tier? Is the economy model producing acceptable results? Did prompt caching reduce repeated input? Are subagents improving results enough to justify their cost? Without this data, model routing becomes guesswork. Do not optimize cost in isolation. A cheaper request that creates more support tickets is not cheaper. A small model that misclassifies 8% of requests may create expensive downstream failures. Track a quality measure appropriate for the feature. For extraction: Field accuracy Missing-field rate Human correction rate For support: Resolution rate Escalation rate User correction rate For agents: Task completion Tool failure rate Retry rate Human intervention Now compare: Cost per run + Quality + Latency + Failure rate That is a much stronger routing signal than price per token alone. A cost-optimized model will sometimes fail. That does not mean every request must start with the expensive model. Use escalation. Economy model ↓ Quality check passes? ↙ ↘ Yes No ↓ ↓ Return Balanced model ↓ Still uncertain? ↙ ↘ No Yes ↓ ↓ Return Frontier model This can keep the common path inexpensive while preserving a stronger path for difficult cases. The quality gate might be: Choose it around the workflow. A document workflow might use several tiers. PDF uploaded ↓ Extract text ↓ Economy model: classify document ↓ Economy model: extract known fields ↓ Validation ↓ Missing ambiguity? ↙ ↘ No Yes ↓ ↓ Save Balanced model: resolve context ↓ Consequential decision? ↙ ↘ No Yes ↓ ↓ Save Frontier model or human review The workflow does not sacrifice intelligence. It spends intelligence where ambiguity increases. The same pattern can apply to support. Customer message ↓ Economy model: intent classification ↓ Deterministic routing ↓ Retrieve account + docs ↓ Balanced model: prepare response ↓ High-risk action requested? ↙ ↘ No Yes ↓ ↓ Reply Frontier reasoning + approval boundary Again, the expensive path is available. It is simply not the default for every message. Do not replace every model path at once. Start with one high-volume feature. Record: Identify which steps require: Run representative inputs through alternative configurations. Compare quality before changing production routing. Start with a narrow percentage of traffic. If the cheaper route is uncertain, move the request upward. Do not stop at token savings. Measure whether the product still completes the job properly. OpenAI's recent builder guide describes several production teams reducing AI costs through smaller models, lower reasoning effort, prompt caching, and architectural changes. It also argues that many workflows no longer need a frontier model at every stage. The exact model choices will continue changing. That makes the architecture behind selection more valuable than any single recommendation. Model pricing will change. New models will arrive. Capabilities will overlap. Reasoning controls will change. Latency will improve. If every feature directly chooses its own provider model, each change becomes a migration project. A central task-aware router gives you a stable product boundary. The feature says: This is the job I need done. The routing layer decides: What is the lowest-cost path that can do it reliably? That is a much healthier economic contract for an AI product. OpenAI, The builder's guide to GPT-5.6 https://openai.com/index/builders-guide-to-gpt-5-6/