Build a Claude vs GPT Router: 12 Steps, 90 Min [2026]

A 12-step, roughly 90-minute tutorial published in October 2026 shows developers how to build a FastAPI router that sends each request to either Anthropic's Claude 5.5 family or OpenAI's GPT-6 family based on task difficulty, then scores quality, latency and cost with a third model acting as judge. The tutorial cites Anthropic's Claude Platform pricing documentation listing Claude Opus 5.5 at $4 per million input tokens and $20 per million output tokens, Claude Sonnet 5.5 at $2 and $10, and Claude Haiku 5.5 at roughly $0.10 and $0.50 with a 1 million token context window and 128,000 token maximum output, against OpenAI's gpt-6-luna at $0.10 input and $0.50 output per million tokens. It cites the 2025 Stack Overflow Developer Survey finding that "OpenAI GPT models were used by 81% of developers for development work in the past year, compared with 43% for Anthropic Claude Sonnet models" as the reason teams route between both vendors rather than standardizing on one.

Every team shipping an AI feature in October 2026 runs into the same wall. Claude Opus https://shattered.io/claude-opus-6-rumors-opus-5-5-cost-cut-2026/ 5.5 handles the hard reasoning tasks beautifully but costs $4 per million input tokens and $20 per million output tokens. GPT-6 Luna handles the easy ones for a tenth of a cent and barely dents a budget. Nobody wants to hardcode one provider and eat the cost of running every request through the expensive model, and nobody wants to hand-pick a model for every single API call either. The fix is a router: a thin service that looks at each request, decides which model it actually needs, sends it there, and logs what happened so you can prove the routing decision was correct. This tutorial builds that router from scratch using the Claude 5.5 family and the GPT-6 family, then adds an evaluation harness that scores quality, latency, and cost side by side so the routing policy is based on data instead of guesswork. It fits alongside the rest of our AI and machine learning coverage https://shattered.io/ai-machine-learning/ , where we track model releases and pricing shifts from both labs as they happen. By the end you will have a working FastAPI service, provider adapters for both vendors, a task classifier, a fallback chain, token-level cost tracking, and a scoring loop that uses a third model as a judge. Twelve steps, roughly 90 minutes if you copy the code blocks as you go. Why a Claude vs GPT router matters right now Anthropic and OpenAI have both split their lineups into tiers that are priced an order of magnitude apart, and the gap is wide enough that routing decisions now show up directly on the monthly invoice. Anthropic’s pricing page lists Claude Opus 5.5 at $4 per million input tokens and $20 per million output tokens, Claude Sonnet 5.5 at $2 and $10, and Claude Haiku 5.5 at roughly $0.10 and $0.50, with Haiku 5.5 carrying a 1 million token context window and a 128,000 token maximum output, according to Anthropic’s Claude Platform pricing documentation https://platform.claude.com/docs/en/about-claude/pricing?38d7aa68 page=4&e17340ce page=7&fcdaa149 page=2&fcdaa149 sort date=desc . On the OpenAI side, the current OpenAI API pricing reference https://developers.openai.com/api/docs/pricing.md lists gpt-6-luna at $0.10 input and $0.50 output per million tokens, with gpt-6-astra and gpt-6.1-sol sitting at higher tiers for harder tasks. Sending every request to the flagship model is the easiest way to burn through a budget. Sending every request to the cheapest model is the easiest way to ship a product that gets reasoning questions wrong. A router splits the difference by profiling each incoming request and only reaching for Opus 5.5 or GPT-6 Astra when the task actually needs that level of capability. Developer adoption patterns back up why both providers matter enough to route between rather than picking one. Stack Overflow reported that “OpenAI GPT models were used by 81% of developers for development work in the past year, compared with 43% for Anthropic Claude Sonnet models,” per the 2025 Stack Overflow Developer Survey https://stackoverflow.co/company/press/archive/stack-overflow-2025-developer-survey/ . Both ecosystems are heavily used, which is exactly the situation a router is built for: pick the right tool per task instead of betting the whole stack on one vendor. Prerequisites: accounts, SDKs, and versions you’ll need Gather these before starting Step 1. Nothing here is optional, and mismatched SDK versions are the single most common reason this kind of project breaks on a fresh machine. - Python 3.12 or newer 3.11 works, 3.13 works, anything older will fail on type hints used below - An Anthropic API key with billing enabled, generated from the Claude Console - An OpenAI API key with billing enabled, generated from the OpenAI developer platform - anthropic Python SDK, version 0.45 or newer - openai Python SDK, version 1.60 or newer - fastapi 0.115 or newer and uvicorn 0.32 or newer for the service layer - pydantic 2.9 or newer for the shared request schema - httpx 0.27 or newer, used internally by both SDKs for async calls - A terminal, roughly 90 minutes, and about $5 to $10 of API credit to run the evaluation harness against a real test set Run the install in one shot so version conflicts surface immediately instead of three steps in: python3 -m venv router-env source router-env/bin/activate pip install "anthropic =0.45" "openai =1.60" "fastapi =0.115" "uvicorn =0.32" "pydantic =2.9" "httpx =0.27" Step 1: Get API keys and set up your environment Create a Claude API key in the Claude Console under Settings then API Keys, and an OpenAI key under API Keys in the OpenAI platform dashboard. Store both as environment variables instead of hardcoding them anywhere near the router logic. A leaked router key is a direct line to your billing account for both providers at once, so treat this file the same way you would treat a database password. .env never commit this file ANTHROPIC API KEY=sk-ant-your-key-here OPENAI API KEY=sk-your-key-here ROUTER LOG LEVEL=info ROUTER DEFAULT BUDGET USD=0.02 Load it with python-dotenv or your process manager’s native env file support. The ROUTER DEFAULT BUDGET USD value matters later: it is the ceiling the router will refuse to cross for a single request unless a caller explicitly overrides it, which is the cheapest insurance policy against a routing bug that accidentally sends every request to the most expensive model available. Step 2: Design a provider-agnostic request schema The whole point of a router is that the caller should never know or care which vendor answered the request. That only works if every adapter speaks the same internal language. Build one Pydantic model that every adapter accepts and every adapter returns, and keep provider-specific fields out of it entirely. python from pydantic import BaseModel from typing import Literal, Optional class RouterRequest BaseModel : prompt: str system: Optional str = None task type: Literal "classify", "generate", "reason", "extract", "chat" = "chat" max tokens: int = 1024 budget usd: Optional float = None require model: Optional str = None escape hatch to force a specific model class RouterResponse BaseModel : text: str provider: str model: str input tokens: int output tokens: int cached input tokens: int = 0 cost usd: float latency ms: int Notice task type is deliberately coarse. You do not need a sophisticated taxonomy to get most of the cost savings from routing. Five buckets, classify, generate, reason, extract, chat, cover the large majority of production traffic and map cleanly onto a tier of models from each provider. Resist the urge to add twenty categories before you have evaluation data proving the split is worth the complexity. Step 3: Build the Claude adapter The adapter’s job is narrow: accept a RouterRequest , call the Anthropic SDK, and return a RouterResponse with real token counts and a computed cost. Anthropic’s October 2026 release notes confirm the current model identifiers for the mid and budget tiers as claude-sonnet-5-5 and claude-haiku-5-5 , per the Claude Platform release notes https://docs.anthropic.com/en/release-notes/api . The flagship tier follows the same naming pattern as claude-opus-5-5 ; confirm the exact string against your own Anthropic console before deploying, since Anthropic occasionally ships a point release under a slightly different string. python import time import anthropic CLAUDE PRICING = { "claude-opus-5-5": {"input": 4.00, "output": 20.00}, "claude-sonnet-5-5": {"input": 2.00, "output": 10.00}, "claude-haiku-5-5": {"input": 0.10, "output": 0.50}, } client = anthropic.Anthropic def call claude model: str, req - dict: start = time.monotonic message = client.messages.create model=model, max tokens=req.max tokens, system=req.system or "", messages= {"role": "user", "content": req.prompt} , latency ms = int time.monotonic - start 1000 usage = message.usage price = CLAUDE PRICING model cost = usage.input tokens / 1 000 000 price "input" + \ usage.output tokens / 1 000 000 price "output" return { "text": message.content 0 .text, "provider": "anthropic", "model": model, "input tokens": usage.input tokens, "output tokens": usage.output tokens, "cached input tokens": getattr usage, "cache read input tokens", 0 or 0, "cost usd": round cost, 6 , "latency ms": latency ms, } Keep the pricing table as data, not as scattered magic numbers inside the function. Anthropic revises prices more often than most teams revise their own code, and a release note like the one above showing prompt cache reads on Sonnet 5.5 dropping from $0.20 to $0.10 per million tokens is exactly the kind of change that should require editing one dictionary, not hunting through every call site. Step 4: Build the GPT-6 adapter The GPT-6 adapter mirrors the Claude adapter in shape but has one wrinkle Claude does not: a long-context pricing cliff. OpenAI bills any request over 272,000 input tokens at roughly double the input rate and 1.5 times the output rate, a threshold documented in the BenchLM OpenAI API pricing breakdown https://benchlm.ai/openai/api-pricing for October 2026. If your adapter does not check for that cliff before the call, a single oversized document upload can quietly double your bill for that request. python from openai import OpenAI GPT PRICING = { "gpt-6-astra": {"input": 10.00, "output": 50.00, "long input": 20.00, "long output": 75.00}, "gpt-6.1-sol": {"input": 2.00, "output": 10.00, "long input": 4.00, "long output": 15.00}, "gpt-6-luna": {"input": 0.10, "output": 0.50, "long input": 0.20, "long output": 0.75}, } LONG CONTEXT THRESHOLD = 272 000 oa client = OpenAI def call gpt model: str, req - dict: start = time.monotonic messages = if req.system: messages.append {"role": "system", "content": req.system} messages.append {"role": "user", "content": req.prompt} response = oa client.chat.completions.create model=model, max tokens=req.max tokens, messages=messages, latency ms = int time.monotonic - start 1000 usage = response.usage price = GPT PRICING model is long = usage.prompt tokens LONG CONTEXT THRESHOLD in rate = price "long input" if is long else price "input" out rate = price "long output" if is long else price "output" cost = usage.prompt tokens / 1 000 000 in rate + \ usage.completion tokens / 1 000 000 out rate return { "text": response.choices 0 .message.content, "provider": "openai", "model": model, "input tokens": usage.prompt tokens, "output tokens": usage.completion tokens, "cached input tokens": getattr usage, "cached tokens", 0 or 0, "cost usd": round cost, 6 , "latency ms": latency ms, } Both adapters now return the exact same shape of dictionary, which is what lets the router compare them like for like a few steps from now. Step 5: Write a task classifier You already have a task type field callers can set manually, but most production traffic arrives without one. The cheapest way to fill the gap is to run a short classification pass on the cheapest model available before touching anything expensive. Haiku 5.5 and GPT-6 Luna both sit at the $0.10 input tier, so either works as the classifier and the cost is close to rounding error. CLASSIFIER PROMPT = """Classify the following request into exactly one category: classify, generate, reason, extract, chat. Reply with only the category word, nothing else. Request: {prompt}""" def classify task prompt: str - str: result = call claude "claude-haiku-5-5", RouterRequestLike prompt=CLASSIFIER PROMPT.format prompt=prompt :2000 , system=None, max tokens=10, label = result "text" .strip .lower valid = {"classify", "generate", "reason", "extract", "chat"} return label if label in valid else "chat" Truncating the prompt to 2,000 characters before classification keeps this step fast and cheap even when the real request is a 50-page document. The classifier only needs enough signal to guess the task type, not the full payload. Step 6: Implement the routing policy With a task type in hand, the routing policy is just a lookup table mapping task type to a preferred model and a cheaper fallback. The table below is a reasonable starting point built from the current pricing tiers; adjust it once your evaluation harness in Step 9 gives you real accuracy numbers for your own prompts. | Task type | Primary model | Cheap fallback | Why | |---|---|---|---| | classify | claude-haiku-5-5 | gpt-6-luna | Low ambiguity, cost dominates | | extract | gpt-6-luna | claude-haiku-5-5 | Structured output, large context window helps on long documents | | chat | claude-sonnet-5-5 | gpt-6.1-sol | Balance of quality and latency for user-facing replies | | generate | gpt-6.1-sol | claude-sonnet-5-5 | Mid-tier reasoning, moderate output length | | reason | claude-opus-5-5 | gpt-6-astra | Multi-step reasoning justifies the premium tier | ROUTING TABLE = { "classify": "anthropic", "claude-haiku-5-5" , "openai", "gpt-6-luna" , "extract": "openai", "gpt-6-luna" , "anthropic", "claude-haiku-5-5" , "chat": "anthropic", "claude-sonnet-5-5" , "openai", "gpt-6.1-sol" , "generate": "openai", "gpt-6.1-sol" , "anthropic", "claude-sonnet-5-5" , "reason": "anthropic", "claude-opus-5-5" , "openai", "gpt-6-astra" , } def choose route req - tuple str, str : if req.require model: provider = "anthropic" if "claude" in req.require model else "openai" return provider, req.require model candidates = ROUTING TABLE req.task type return candidates 0 The require model escape hatch matters more than it looks. There will always be a caller, usually someone debugging a weird output, who needs to pin a request to one exact model and bypass the policy entirely. Build that door in from day one instead of bolting it on after the third person asks for it. Step 7: Add fallback chains and retry logic The routing table above already lists a second candidate per task type. Wire it up so a failure on the primary model, a timeout, a rate limit, a 500 from the provider, falls through to the fallback instead of returning an error to the caller. This is the difference between a router that is a nice cost optimization and one that actually improves reliability. python import asyncio ADAPTERS = {"anthropic": call claude, "openai": call gpt} async def route with fallback req - dict: provider, model = choose route req candidates = provider, model + c for c in ROUTING TABLE req.task type if c = provider, model last error = None for provider name, model name in candidates: try: adapter = ADAPTERS provider name return await asyncio.wait for asyncio.to thread adapter, model name, req , timeout=25.0 except Exception as exc: last error = exc continue raise RuntimeError f"All routing candidates failed: {last error}" A 25-second timeout per candidate keeps a hung provider from stalling the whole request chain. If both Claude and GPT-6 time out, the caller gets a clear error instead of waiting a minute to find out nothing worked. Step 8: Track tokens and cost per request Every adapter already returns cost usd , so the remaining work is persistence, not calculation. Log every routed request to a table with the task type, the model chosen, the cost, and the latency. This log is what eventually tells you whether the routing table in Step 6 is actually saving money or just moving it around. python import sqlite3 import json from datetime import datetime, timezone conn = sqlite3.connect "router log.db" conn.execute """ CREATE TABLE IF NOT EXISTS requests id INTEGER PRIMARY KEY AUTOINCREMENT, timestamp TEXT, task type TEXT, provider TEXT, model TEXT, input tokens INTEGER, output tokens INTEGER, cached input tokens INTEGER, cost usd REAL, latency ms INTEGER """ def log request req, result: dict - None: conn.execute "INSERT INTO requests timestamp, task type, provider, model, input tokens, " "output tokens, cached input tokens, cost usd, latency ms VALUES ?,?,?,?,?,?,?,?,? ", datetime.now timezone.utc .isoformat , req.task type, result "provider" , result "model" , result "input tokens" , result "output tokens" , result "cached input tokens" , result "cost usd" , result "latency ms" , , conn.commit SQLite is enough for a single-instance router handling a few thousand requests a day. Swap it for Postgres once you have more than one router process writing to the same log, since SQLite’s file-level locking will start to bottleneck concurrent writers. Step 9: Build the evaluation harness The routing table in Step 6 is a hypothesis, not a fact. The evaluation harness is what turns it into a fact by running the same prompt through multiple models in parallel and recording how each one did on quality, latency, and cost. Build a test set of 20 to 50 real prompts pulled from your own traffic, grouped by task type, and run the full set through every candidate model whenever you change the routing table. python async def evaluate prompt prompt: str, task type: str, models: list tuple str, str - list dict : req = RouterRequest prompt=prompt, task type=task type async def run one provider, model : adapter = ADAPTERS provider result = await asyncio.to thread adapter, model, req result "prompt" = prompt return result return await asyncio.gather run one p, m for p, m in models async def run eval suite test set: list dict - list dict : all results = for case in test set: candidates = ROUTING TABLE case "task type" results = await evaluate prompt case "prompt" , case "task type" , candidates all results.extend results return all results Running candidates with asyncio.gather keeps the harness fast: every prompt hits both providers at once instead of waiting for Claude to finish before calling GPT-6. On a 40-prompt test set across five task types, this finishes in two or three minutes instead of fifteen. Step 10: Score responses with an LLM-as-judge Latency and cost are easy to measure automatically. Quality is not, which is why most evaluation harnesses use a third model as a judge rather than relying on exact-match string comparison. Use the strongest model available, Opus 5.5 or GPT-6 Astra, purely as a judge, never as a candidate in the same comparison, and ask it to score on a fixed rubric so results stay comparable across runs. JUDGE PROMPT = """You are grading an AI response for correctness and helpfulness. Original prompt: {prompt} Response to grade: {response} Score the response from 1 to 5, where 5 means fully correct and directly useful, and 1 means incorrect or unusable. Reply with only the number.""" def judge response prompt: str, response: str - int: req = RouterRequestLike prompt=JUDGE PROMPT.format prompt=prompt, response=response , system=None, max tokens=5, result = call claude "claude-opus-5-5", req try: return int result "text" .strip 0 except ValueError, IndexError : return 0 Run the judge on every response from Step 9, not just the ones you suspect are wrong. Judges disagree with human graders more often than teams expect, so spot-check a random 10% of judged responses by hand every time you refresh the test set, and treat the judge’s score as a strong signal rather than ground truth. Step 11: Generate a quality-per-dollar report With cost, latency, and a judge score attached to every result from Step 9 and Step 10, the harness can finally answer the question that matters: which model gives you the most quality per dollar for each task type. This is the number that should actually drive the routing table in Step 6, not the raw benchmark scores vendors publish in launch blog posts. python from collections import defaultdict def build report scored results: list dict - dict: grouped = defaultdict list for r in scored results: key = r "task type" , r "provider" , r "model" grouped key .append r report = {} for key, rows in grouped.items : avg score = sum r "score" for r in rows / len rows avg cost = sum r "cost usd" for r in rows / len rows avg latency = sum r "latency ms" for r in rows / len rows quality per dollar = avg score / avg cost if avg cost 0 else 0 report key = { "avg score": round avg score, 2 , "avg cost usd": round avg cost, 6 , "avg latency ms": round avg latency, 1 , "quality per dollar": round quality per dollar, 1 , "n": len rows , } return report Here is what a typical report looks like on a 40-prompt test set split across the five task types, using mock scores for illustration since your actual numbers will depend entirely on your own prompts: | Task type | Model | Avg score 1-5 | Avg cost | Avg latency | Quality per dollar | |---|---|---|---|---|---| | classify | claude-haiku-5-5 | 4.6 | $0.00009 | 410ms | 51,111 | | classify | gpt-6-luna | 4.3 | $0.00011 | 380ms | 39,090 | | chat | claude-sonnet-5-5 | 4.7 | $0.0021 | 890ms | 2,238 | | chat | gpt-6.1-sol | 4.5 | $0.0019 | 920ms | 2,368 | | reason | claude-opus-5-5 | 4.9 | $0.014 | 2,100ms | 350 | | reason | gpt-6-astra | 4.8 | $0.018 | 2,350ms | 267 | Output like this is what lets you defend a routing decision to a finance team instead of pointing at a vendor’s marketing page. It also tends to surprise people: in this illustrative run, GPT-6.1 Sol edges out Claude Sonnet 5.5 on quality per dollar for chat despite a slightly lower average score, because it is also slightly cheaper per call. Step 12: Wrap it in a FastAPI service The last step turns the functions above into something other services can actually call. Keep the endpoint thin: validate the request, call the router, log the result, return the response. python from fastapi import FastAPI, HTTPException app = FastAPI title="Claude vs GPT Router" @app.post "/v1/route", response model=RouterResponse async def route request req: RouterRequest : if req.task type == "chat" and not req.require model: req.task type = classify task req.prompt or req.task type try: result = await route with fallback req except RuntimeError as exc: raise HTTPException status code=502, detail=str exc budget = req.budget usd or 0.02 if result "cost usd" budget: result "warning" = f"Request exceeded budget of ${budget}" log request req, result return result @app.get "/health" def health : return {"status": "ok"} Start it with uvicorn router:app --reload --port 8080 and send a test request with curl to confirm the whole chain works end to end before moving on to real traffic. curl -X POST http://localhost:8080/v1/route \ -H "Content-Type: application/json" \ -d '{"prompt": "Summarize the key risks in this contract clause.", "task type": "reason"}' A healthy response looks like this, with the model, cost, and latency filled in from the real API call rather than placeholders: { "text": "The clause shifts liability for...", "provider": "anthropic", "model": "claude-opus-5-5", "input tokens": 142, "output tokens": 310, "cached input tokens": 0, "cost usd": 0.013536, "latency ms": 1894 } If you need the router to sit in front of an agent framework rather than raw chat traffic, the same pattern extends cleanly. Readers who already followed our guide to building a secure MCP server for Claude https://shattered.io/build-secure-mcp-server-claude-2026/ can expose this router as one more MCP tool, letting Claude Code or any other MCP client call it directly instead of talking to Anthropic’s API alone. Common pitfalls to avoid - Hardcoding model strings everywhere. When Anthropic or OpenAI ship a point release, you want to change one pricing dictionary, not grep the entire codebase for a string. - Ignoring the 272K-token long-context cliff. A single oversized GPT-6 request can silently double its cost if your adapter does not check the threshold before billing. - Trusting published benchmarks over your own evaluation data. A model that wins on a general benchmark can still lose on quality per dollar for your specific prompts, which is why Step 9 through Step 11 exist. - Skipping prompt caching. Anthropic’s cache read pricing on Sonnet 5.5 sits at a fraction of the base input rate; a router that resends the same system prompt uncached on every call is leaving money on the table for no benefit. - Routing purely on cost with no latency floor. The cheapest model is not always fast enough for a user-facing chat endpoint with a 2-second SLA, even if it wins on quality per dollar in a batch evaluation. - No timeout on fallback chains. Without the 25-second cap from Step 7, one slow provider can block every downstream request behind it. - Comparing judge scores across different judge models. Switching the judge from Opus 5.5 to GPT-6 Astra mid-evaluation invalidates the comparison; keep one judge model fixed for the life of a test set. Troubleshooting guide These are the failures that show up most often once the router handles real traffic instead of a handful of manual test calls. - 401 Unauthorized from Anthropic or OpenAI. The environment variable is not loaded into the process, usually because the .env file was placed in the wrong directory relative to where uvicorn was launched. - 429 Too Many Requests during the evaluation harness. Running 40 prompts across five task types fires dozens of concurrent calls; add a semaphore limiting concurrent requests per provider to 5 or 10. - Router always falls through to the fallback model. Check that the primary adapter is not silently throwing inside the try block in Step 7; add logging on the except branch instead of swallowing the exception. - Cost numbers look too high. Confirm the pricing dictionary matches the current per-million-token rate, not a per-thousand-token rate left over from an older pricing era. - Judge scores all come back as 0. The judge’s reply format drifted from a bare number to a sentence; tighten the judge prompt or parse with a regex instead of taking the first character. - SQLite “database is locked” errors. You have more than one router process writing to the same file; move to Postgres or set conn.execute "PRAGMA journal mode=WAL" as an interim fix. - Classifier mislabels tasks as “chat” constantly. The fallback in classify task defaults to “chat” on any unexpected output; log the raw classifier output to see whether it is actually failing or just returning something outside the five valid labels. - Long-context GPT-6 calls return unexpectedly high costs. Verify the adapter is reading usage.prompt tokens correctly and applying the long-context rate only above 272,000 tokens, not above some other threshold copied from an older model generation. - asyncio.gather hangs with no error. One of the adapter calls inside asyncio.to thread is blocking without a timeout; wrap each call in asyncio.wait for as shown in Step 7, not just the top-level gather. Advanced tips for production routing Once the basic router works, a few upgrades pay off quickly in production. Streaming matters first: wire both adapters to stream tokens back to the caller instead of waiting for the full response, which cuts perceived latency on chat-style requests dramatically even though the total cost and token count stay the same. Semantic caching is the second upgrade; hashing exact prompts only catches literal repeats, so pairing the router with an embedding-based cache catches near-duplicate requests too, which matters a lot for support-bot style traffic where the same question gets asked a dozen different ways. A multi-armed bandit routing policy is worth adopting once you have a few weeks of evaluation data. Instead of a static lookup table, the router allocates a small percentage of traffic to the non-default model for each task type on an ongoing basis, which keeps the quality-per-dollar numbers from Step 11 fresh as providers ship silent model updates. This also catches the case where a vendor quietly changes model behavior between point releases without announcing it, something that happens more often than either provider admits. Finally, treat cost alerts as a first-class feature, not an afterthought. A router that silently absorbs a routing bug sending every “classify” request to Opus 5.5 for three days will not announce the mistake, your invoice will. Wire the cost usd field from every logged request into a rolling daily total with a Slack or email alert if it crosses a threshold you set deliberately, and you will catch that kind of regression within hours instead of at the end of the billing cycle. Teams already running an AWS Bedrock AgentCore agent https://shattered.io/deploy-aws-bedrock-agentcore-agent-2026/ can reuse the same CloudWatch alarm pattern for this router’s cost log table instead of building alerting from scratch. How this compares to picking one model and sticking with it It is worth asking whether a router is overkill for a smaller project, and sometimes it is. If your traffic is overwhelmingly one task type, say a support bot that only ever answers short factual questions, a single model like Claude Haiku 5.5 or GPT-6 Luna covers the job without the added moving parts of a router. The router earns its complexity once traffic spans multiple task types with genuinely different quality requirements, which is most production AI features once they grow past a single use case. Our earlier benchmark coverage of Claude Opus 5.5 against GPT-6 Astra on Terminal-Bench https://shattered.io/claude-opus-5-5-vs-gpt-6-astra-terminal-bench-2026/ shows the gap between flagship models narrows on some tasks and widens sharply on others, which is the exact pattern that makes static “pick one model” decisions risky. A router does not require picking a winner once; it lets the evaluation data from Step 9 through Step 11 keep making that decision for you, task type by task type, as both vendors keep shipping updates. The complete working project Here is how the pieces from all twelve steps fit together as a single project layout you can drop into a repository: router-project/ ├── .env API keys, never committed ├── requirements.txt ├── router.py FastAPI app, Step 12 ├── adapters/ │ ├── claude adapter.py Step 3 │ └── gpt adapter.py Step 4 ├── classifier.py Step 5 ├── routing table.py Step 6 ├── fallback.py Step 7 ├── cost log.py Step 8 ├── eval/ │ ├── test set.json your own 20-50 real prompts, grouped by task type │ ├── harness.py Step 9 │ ├── judge.py Step 10 │ └── report.py Step 11 └── router log.db generated at runtime Run the whole evaluation suite with a single entry point once the test set is populated with your own prompts: python -m eval.harness --test-set eval/test set.json --judge claude-opus-5-5 That single command runs every prompt against every candidate model for its task type, scores each response with the judge, and prints the quality-per-dollar table from Step 11. Re-run it any time either provider ships a new model, since a new release can reshuffle which model wins which task type without any code changes on your end, just a new row in the pricing dictionaries from Step 3 and Step 4. Teams coming from a pure coding-assistant comparison, like the one in our GitHub Copilot vs Cursor breakdown https://shattered.io/github-copilot-vs-cursor-4-models-8-days-2026/ , will recognize the same underlying lesson here: picking an AI tool once and never re-measuring it is how teams end up overpaying for months without noticing. The router and harness built in this tutorial exist specifically to keep that measurement running automatically instead of depending on someone remembering to check. One more cost lever worth building into the pricing dictionaries from the start: Anthropic’s Claude Platform release notes show prompt cache pricing moves independently of base input and output pricing, and the Economize Cloud pricing breakdown for OpenAI shows the same pattern on cached tokens for GPT-6 models. If your router reuses the same system prompt across many requests, which most production routers do, caching alone can cut your effective input cost by more than half without touching the routing table at all. Our coverage of Anthropic’s 75% API price cut on Claude Haiku 5.5 https://shattered.io/claude-haiku-5-5-api-price-cut-75-percent-2026/ is a good example of how quickly these numbers move and why the pricing dictionary in Step 3 needs to stay easy to update. Frequently asked questions Do I need accounts with both Anthropic and OpenAI to follow this tutorial? Yes. The router’s entire value comes from having two real providers to choose between. A single $5 to $10 credit load on each account is enough to build and test everything in this guide. Which model should I default to if I only want to build a simpler version of this? Start with Claude Sonnet 5.5 or GPT-6.1 Sol as a single default model, then add the second provider once you have real traffic data showing which task types would benefit from a cheaper or stronger alternative. How often should I re-run the evaluation harness? Re-run it whenever either provider ships a new model release, and on a fixed schedule, monthly is reasonable, even if no release has been announced, since providers sometimes update model behavior quietly between named releases. Is SQLite good enough for the cost log in production? For a single router instance handling low to moderate traffic, yes. Once you run multiple router instances behind a load balancer, move the cost log to Postgres to avoid write-lock contention. Can I add a third provider like Gemini to this same router? Yes. The adapter pattern from Step 3 and Step 4 is designed to be provider-agnostic. Add a third adapter that returns the same RouterResponse shape, add its pricing to a new dictionary, and extend the routing table with its model identifiers. What happens if both the primary and fallback model fail? The route with fallback function in Step 7 raises a RuntimeError after exhausting every candidate for that task type, which the FastAPI endpoint in Step 12 converts into a 502 response. Add a third candidate per task type if your reliability requirements call for it. Does using an LLM as a judge introduce bias toward that judge’s own vendor? It can. Mitigate this by never including the judge model itself as a candidate in the same comparison, and by spot-checking a sample of judged responses by hand, as noted in Step 10. Is a 272,000-token threshold the same for every GPT-6 model? Based on current OpenAI pricing documentation, the long-context threshold applies across the GPT-6 family, but confirm the exact figure against the live pricing page before launching, since providers do adjust these thresholds between releases.