{"slug": "amazon-replaced-sonnet-4-6-with-a-4b-decision-model", "title": "Amazon replaced Sonnet 4.6 with a 4B decision model", "summary": "Two Amazon engineers replaced Claude Sonnet 4.6 with Strands Decider, an open 4B decision model, in one of Amazon's largest internal ticketing systems and report the same labelling quality at 98 percent less cost, according to their write-up. A 6,724-token ticket that costs 2.2 cents on Sonnet at $3 per million input and $15 per million output tokens costs about $0.0005 on an ml.g5.xlarge instance-hour priced at $1.408, a 44-times reduction. The engineers de-risked the swap with a three-way AWS AppConfig setting (BEDROCK, DUAL, DECIDER) that allowed a 45-second rollback, and warned that reordering request keys moved output probabilities by up to 0.19, so keys are now sorted on every request.", "body_md": "TLDR: Two Amazon engineers swapped Claude Sonnet 4.6 for Strands Decider, an open 4B decision model, in one of the company’s largest internal ticketing systems, and report the same labelling quality at 98 percent less cost. It enables a category of work that has been paying frontier-model prices for a job that never needed sentence generation. The difference between this and a cheaper language model is that a decision model cannot write anything at all: it returns a probability for each option in a fixed list, plus a confidence score.\n\n## What was actually running\n\nThe system labelled millions of internal tickets into 20 category types: customer escalations, capacity requests, automated alarms, security findings. An agent sent every ticket through Sonnet 4.6 with a long system prompt and got back JSON with a label and reasoning.\n\nThat is a classification problem wearing a chat model’s clothes. The output was always one of twenty known labels, but the model was re-reading a prompt longer than the ticket itself and generating an explanation nobody stored.\n\n## The swap, and the numbers\n\nThey started with the 2B Decider and moved to the E4B Gemma 4 build, which did better on a held-out set of tickets. Then they re-tuned the confidence threshold at which a label gets applied.\n\nThe arithmetic is worth sitting with. At $3 per million input tokens and $15 per million output, a 6,724-token ticket costs 2.2 cents. At $1.408 for an `ml.g5.xlarge` instance-hour, the same ticket costs about $0.0005. That is 44 times cheaper.\n\n## How they de-risked it\n\nThey did not flip a switch. The labelling Lambda reads a three-way setting from AWS AppConfig on every message:\n\n- \n**BEDROCK:** Sonnet labels every ticket, Decider is never called.\n- \n**DUAL:** both run in parallel, Sonnet’s answer is applied, and every disagreement is logged.\n- \n**DECIDER:** Decider labels everything, Bedrock is not called.\n\nRolling back was a 45-second config change. Running both on live traffic, then reading every disagreement, told them more than any offline evaluation would have.\n\nThe deployment itself is ordinary: ticket events land in SQS, a Lambda reads the ticket and calls a SageMaker endpoint, CloudWatch holds the logs, the stack is defined in CDK with Beta, Gamma and Prod stages. The endpoint’s network is isolated with no outbound access, and weights are copied in from S3 at start-up.\n\n## The part that will bite anyone copying this\n\nTwo constraints, both from the write-up.\n\n**Label descriptions have to fit a 4,096-token window.** Production’s label descriptions alone were longer than that. The engineers rewrote all 20 labels as one- or two-sentence criteria, sent them as the options of a single choice question, and added a “none” option. On long tickets, the model drops the oldest comments to fit.\n\n**Key order changes the answer.** “Sending the same content with keys in a different order moved the output probabilities by up to 0.19.” They now sort keys on every request. If you serve a decision model behind an API and compare runs without sorting, you are measuring your serializer.\n\n## When this does not work\n\nThe engineers name the boundaries themselves, and they are narrow:\n\n- The task must be a **closed choice from a known catalogue** .\n- The volume must be **high** .\n- The **latency of the old path must matter** .\n- The outputs must not need explanation text.\n- The catalogue must not change weekly.\n- The useful context must fit into a few thousand tokens.\n\nBreak any of those and you are back to a language model, and the likely explanation is that you were paying for reasoning you were not using.\n\n## Why this matters beyond Amazon\n\nDecision models became a recognised class only this autumn, after TypeSafe AI’s Jev, and the pitch has been mostly about latency. This is the first production account with token counts, instance hours, a rollback plan and a failure mode that includes the words “keys in a different order”.\n\nThe lesson for anyone running a classification agent on a frontier API is not that a 4B model is magic. It is that the frontier model was doing two jobs, and only one of them was ever necessary. Strip out the generation, keep the decision, and the bill for a million tickets goes from a salary to a rounding error.\n\nIf you have a phone, a 24GB card, or a spare A10G, the experiment is a weekend: export a thousand labelled examples, build a choice question with the labels as options, and see whether a 4B decision model with a confidence threshold beats the prompt you are paying for.", "url": "https://wpnews.pro/news/amazon-replaced-sonnet-4-6-with-a-4b-decision-model", "canonical_source": "https://tokenstead.ai/guides/strands-decider-tickets", "published_at": "2026-11-10 13:03:58+00:00", "updated_at": "2026-11-10 13:21:59.658330+00:00", "lang": "en", "topics": ["artificial-intelligence", "machine-learning", "ai-infrastructure", "ai-tools"], "entities": ["Amazon", "Claude Sonnet 4.6", "Strands Decider", "Gemma 4", "AWS AppConfig", "SageMaker", "TypeSafe AI", "Jev"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/amazon-replaced-sonnet-4-6-with-a-4b-decision-model", "markdown": "https://wpnews.pro/news/amazon-replaced-sonnet-4-6-with-a-4b-decision-model.md", "text": "https://wpnews.pro/news/amazon-replaced-sonnet-4-6-with-a-4b-decision-model.txt", "jsonld": "https://wpnews.pro/news/amazon-replaced-sonnet-4-6-with-a-4b-decision-model.jsonld"}}