Exowatt: Fine-Tuned Qwen3 Beats Claude Opus 4.7 at 49x Lower Cost on Its Own Tasks

AWS published a customer write-up on October 27, 2026 claiming that two small fine-tuned Qwen3 models beat Claude Opus 4.7, Sonnet 4.6 and Haiku 4.5 on narrow tasks at Exowatt, a maker of solar-plus-thermal power systems for data centers, at up to 49x lower cost. It is a vendor-published case study with a company-built test set, so read the numbers as strong evidence for a pattern, not as a general ranking. Here is what AWS reports, how the method works, and what to verify before copying it.

This is the same shape as the Coinbase result, where a fine-tuned Qwen3.5 9B beat Opus 4.5 on fraud, now with published cost tables.

Quick reference #

| Models | Qwen3-1.7B (edge), Qwen3-14B (cloud), LoRA supervised fine-tuning | 
| Best result | 0.911 macro-F1 (edge) and 0.975 (cloud) | 

| Cost | About $0.60 vs $29.26 per 1,000 answers for Opus 4.7 (cloud); $0 metered at the edge | | Caveat | Vendor-published, custom test sets, narrow tasks |

The problem: AI on physical infrastructure #

Exowatt's energy management system produces alarms continuously across a growing fleet: out-of-spec hardware, dropped connections, out-of-range voltages. Humans cannot watch thousands of sites. AWS lists three reasons a frontier model did not fit: the task is narrow and proprietary, it must run at the edge with no GPU and sometimes no internet, and sending every decision to a frontier API does not scale economically.

The pipeline #

Exowatt used SageMaker AI serverless model customization, which supplies ready-made fine-tuning recipes from the SageMaker Public Hub and allocates capacity on demand, so the team did not choose GPU instance types or build training containers. Artifacts go to the SageMaker Model Registry. The edge model is quantized to GGUF and served with llama.cpp in a Kubernetes pod on a site controller; the cloud model is served through Amazon Bedrock Custom Model Import, billed at $6.86 per hour only while active. Paraphrasing of training data used Qwen3-VL-235B and Claude Sonnet 4.6 on Bedrock. For the broader SageMaker training story see our posts on SageMaker training sessions and the multi-turn RL Wordle walkthrough.

Use case 1: edge alarm triage with Qwen3-1.7B #

The model reads an alarm plus site telemetry and returns one of a small set of classifications with cited evidence, or abstains. AWS says prompting Qwen3-8B plateaued at 0.735 macro-F1, and thirteen fine-tuned 8B candidates across twelve dataset versions never beat that baseline, while a fine-tuned 1.7B model with more varied data did.

Training data was small: 2,522 examples, built from 684 verified decisions on generated site scenarios plus 1,838 certified paraphrases that kept each decision fixed. The paraphrases cost about $3.72 and were kept only if deterministic checks confirmed decisions, labels, citations and numbers were unchanged (1,838 of 2,218 passed). One fault family was held out of training and 320 cases were reserved for testing. Training took about 32 minutes for roughly $20 to $25. The 8-bit model is about 1.71 GiB and runs offline in a 6 GiB llama.cpp pod on the existing CPU server.

The untuned base 1.7B failed the validity gate, so it has no score. These are AWS's figures.

Use case 2: cloud energy-market analysis with Qwen3-14B #

Each site decides how much stored energy to hold and when to release it, so the model reads a six-hour packet of market data (about 90 rows of prices, forecasts and grid notices) and returns one of five market events in about four seconds, citing data rows. Exowatt trained on public ERCOT data: five years, about 163 million rows across 12 tables, rule-labeled into 25,000 examples (22,000 train, 1,000 validation, 2,000 test) and expanded to 86,281 rows by having Claude Sonnet 4.6 rewrite the prose while labels and numbers stayed fixed.

Training was one LoRA epoch (rank 16, alpha 32): about 8.5 hours of managed fine-tuning, with 23 jobs and about 155 hours in six days overall. The Bedrock import took about 14 minutes.

AWS reports a +0.35 macro-F1 gain over Opus 4.7 (95% confidence interval 0.324 to 0.371) and 0.968 on a second, later test set. Costs are Bedrock list prices times measured tokens per answer.

What actually drove the gains #

AWS highlights three lessons that are more transferable than the headline:

1. Data variety beat volume and epochs. On alarm triage, the 684 verified decisions alone scored 0.584; the paraphrase-expanded set reached 0.911. A control that trained 11 epochs on the original 684 rows reached only 0.879.
2. Paraphrase diversity mattered. Three rewrites per answer lifted the cloud model's macro-F1 from 0.950 to 0.975.
3. Grounding rules made autonomy safer. Both models return strict JSON citing input rows, and an evidence fence drops any claim whose source, field or value does not literally match the input. The edge model abstains when evidence is insufficient.

The deployment pieces, explained #

Two open-source components do the heavy lifting once training is done. Qwen3 provides the base models; the 1.7B and 14B sizes are small enough that the tuned result is cheap to host. llama.cpp runs the quantized GGUF file on a plain CPU, which is why the edge model needs no GPU and keeps working when a site loses connectivity. Quantizing to 8 bits shrinks the 1.7B model to about 1.71 GiB, so it fits in a 6 GiB pod next to the other site software.

The cloud model takes a different route: Bedrock Custom Model Import loads the merged checkpoint as a managed model that costs nothing while idle. AWS says Exowatt added a scheduled keep-warm call using Lambda and EventBridge, which is a practical detail worth copying, since imported models can take time to start when cold.

Timeline and effort #

The headline speed claim is that Exowatt moved a customized model from experimentation to production in nine days, versus what AWS says could take months with custom containers and provisioned GPUs. The energy-market model went from base model to serving in the cloud service in six days. That speed comes mostly from removing capacity requests: with serverless training, up to four jobs ran at once and each experiment started immediately. Whether your own team sees the same gain depends on how much time you currently lose to GPU provisioning and recipe building.

How far to trust the comparison #

Reasons for caution, none of which AWS hides but all of which matter:

• Vendor and customer publication. The post is on AWS's blog and promotes AWS services.
• The test set is Exowatt's. Frontier models are scored on a task they were not trained on, with prompts we cannot inspect; a fine-tuned model has seen the task's distribution.
• Narrow, closed-set tasks. Classification into a fixed label set is where small tuned models do best. Open-ended work is different.
• Held-out design is partial. One fault family was held out for edge; the cloud model's test data comes from the same market source as training, though a later second test set held up at 0.968.
• Cost accounting. The $0 edge figure excludes fine-tuning, data work and engineering time, and the 49x figure compares against Opus 4.7 specifically.

A checklist to reproduce the approach on your own task #

1. Define a closed output. Write down the exact label set or JSON schema, including an explicit abstain option, before collecting data.
2. Build verified seeds first. Exowatt started from 684 verified decisions, not from raw logs. A few hundred trustworthy examples beat thousands of noisy ones.
3. Augment with rewrites, then filter. Paraphrase the prose while holding labels and numbers fixed, and keep only rewrites that pass deterministic checks. Exowatt kept 1,838 of 2,218.
4. Hold out something the model has never seen. A whole fault family, a later time period, or a second test set will tell you more than a random split.
5. Benchmark against two or three frontier models at their list prices. Report quality, validity and cost per thousand decisions together.
6. Enforce grounding at runtime. Require citations to input fields and drop anything uncited before it reaches a person or an actuator.
7. Ship where the work happens. If connectivity is unreliable, quantize and run on CPU; if load is bursty, use an imported managed model that costs nothing while idle.

What builders can take from it #

• Fine-tune when the task is narrow, repeated and measurable. Alarm triage, ticket routing and fraud scoring fit; general assistants do not.
• Invest in data, not just model size. Certified paraphrases with deterministic checks are cheap and effective.
• Plan for where the model runs. Offline and CPU-only at the edge, elastic in the cloud, from one pipeline.
• Keep a frontier baseline. Exowatt's evaluation against three Claude models is what makes the result credible; repeat that step for your own task. Our guide torunning a small model plus a cloud model together covers a hybrid alternative.

Related reading #

Figures are as published by AWS on October 27, 2026 and have not been independently reproduced.