AWS Shows a 4B Agent Matching Its GLM-5.3 Teacher: How SeqKD Distillation Works

AWS published a November 2, 2026 walkthrough showing a Qwen3.5-4B shopping agent fine-tuned via sequence-level knowledge distillation (SeqKD) reached an Absolute Success Rate of 0.541 on held-out ShoppingBench problems, matching its teacher GLM-5.3 on Amazon Bedrock, which scored 0.540 when decoded greedily. The recipe used 850 teacher trajectories sampled twice per training problem from 426 training problems, rejection-sampled to about 2,600 training rows and 130 validation rows, with whole problems held out at roughly five percent to avoid leakage. AWS notes the results come from a single vendor-written run, but the approach needs only the teacher's text, with no logits or reinforcement-learning infrastructure, making it usable with closed API-only frontier models.

A small open model can learn to act like a big one, at least on a narrow task. In a walkthrough published on November 2, 2026 https://aws.amazon.com/blogs/machine-learning/build-efficient-agents-with-knowledge-distillation-on-amazon-sagemaker-ai/?ref=explainx , AWS shows a Qwen3.5-4B shopping agent reaching an Absolute Success Rate of 0.541 on held-out ShoppingBench problems. Its teacher, GLM-5.3 on Amazon Bedrock, scored 0.540 when decoded greedily. This is a vendor-written tutorial for a vendor product, and the results come from a single run. Still, it includes real numbers, a clear recipe and honest limits, which makes it useful for anyone paying frontier prices for an agent loop. Here is what it says and how to judge it. The problem: agents bill you per step An agentic task is a loop, not a call. AWS's example is a shopper who asks for "a black waist bag under twenty dollars from a shop with at least 4.5 stars." The agent searches, reads results, refines, verifies and decides. Each step is a separate billed call to the frontier model, and one completed task can take many calls, so per-task cost compounds with usage. The usual fix is a smaller model that is faster and cheaper to host. The catch is quality. Knowledge distillation tries to close that gap by training a small "student" to reproduce a large "teacher's" behavior. What SeqKD is Sequence-level knowledge distillation is the simplest form. The teacher generates answers, and the student is fine-tuned on those answers with supervised learning. The original idea predates today's agents, and you can read the roots in the sequence-level distillation paper https://arxiv.org/abs/1606.07947?ref=explainx . Its big practical advantage, per AWS: it needs only the teacher's text, with no logits and no reinforcement-learning infrastructure. That matters because with a closed, API-only teacher you cannot read logits or fine-tune it. AWS points to the DeepSeek-R1 distilled models as a well-known example, where reasoning traces from a large model were used to fine-tune smaller Qwen and Llama students see the DeepSeek-R1 paper https://arxiv.org/abs/2501.12948?ref=explainx . For agents the "sequence" is a trajectory: reasoning, tool calls and the observations they return, step by step. For a refresher on the idea in general, see our guide to AI distillation and knowledge transfer https://explainx.ai/blog/what-is-ai-distillation-knowledge-transfer-fable-5-2026 and the broader fine-tuning guide https://explainx.ai/blog/what-is-fine-tuning-llm-complete-guide-2026 . The recipe AWS used Funnel pouring a large stream into a small one, illustrating distilling a frontier agent into a 4B model 1. Pick a benchmark with a programmatic grader. ShoppingBench has 750 synthesized shopping problems, 250 each for product, shop and voucher intents, with a hidden gold answer and constraints. AWS split them once into 426 training and 324 held-out problems product 103, shop 105, voucher 116 using fixed indices so the sets do not overlap. The catalog holds about 2.7 million products. 2. Generate trajectories with the teacher. GLM-5.3 on Amazon Bedrock, with reasoning on and temperature 1.0, acted as the agent. AWS sampled it twice per training problem, giving 850 trajectories. 3. Keep only the solved runs. ShoppingBench's own scorer filters the trajectories, a step called rejection sampling. The teacher only has to be right often enough that its solved runs are worth copying. 4. Encode per step. Each trajectory becomes one row per step. The user turn carries the system prompt and the conversation so far. The assistant turn carries that step's reasoning and the tool call. Loss falls on the assistant turn only. The solved trajectories expanded to about 2,600 training rows and 130 validation rows. 5. Split by problem, not by row. Many rows share one problem, so a row-level split would leak near-identical context into validation. AWS held out whole problems, about five percent. 6. Compress observations. Tool outputs dominate token counts. AWS advises keeping only the fields the next decision needs id, price, rating, the constraints and capping results per search, so rows fit the 16,384-token limit. 7. Fine-tune with LoRA on serverless SFT. The student is Qwen3.5-4B with LoRA rank 64, alpha 256, three epochs, global batch size 64 and learning rate 0.0001, run through SageMaker AI model customization. The results AWS reports Absolute Success Rate, the share of held-out problems solved, with missing trajectories counted as failures. A problem counts as solved only when the recommended products meet every stated constraint. | Model | Product | Shop | Voucher | Average | |---|---|---|---|---| | Base Qwen3.5-4B no fine-tuning | 0.456 | 0.314 | 0.121 | 0.297 | | Distilled student SeqKD | 0.563 | 0.638 | 0.422 | 0.541 | | GLM-5.3 teacher, sampled temp 1.0 | 0.553 | 0.619 | 0.362 | 0.511 | | GLM-5.3 teacher, greedy temp 0 | 0.544 | 0.638 | 0.439 | 0.540 | | Medium-sized frontier model | 0.583 | 0.667 | 0.388 | 0.546 | The student gained about 24 points on average over the untrained base model, and matched its teacher within run-to-run noise. AWS explains why the student can edge past the sampled teacher: rejection sampling trains it on the teacher's solved runs, not its average output, so it imitates a best-of slice. One caution: AWS says the numbers come "from a single run with a LoRA student, so expect a few points of run-to-run variation." Differences of 0.001 to 0.006 between the student and the teacher or frontier model are not meaningful. The fair reading is "roughly equal," not "better." Reasoning in the targets matters The most useful finding is an ablation. Training the same student on the same trajectories with reasoning removed, leaving only bare tool calls, dropped average success from 0.541 to 0.415. Keeping the reasoning added about 13 points on average, and the gain held on every intent. Both versions still beat the base model, so even tool-calls-only distillation helps. AWS's conclusion: "the reasoning between the actions isn't narration to discard, it is part of the behavior that solves the task." If you build your own distillation set, this is the lesson to copy: keep the teacher's reasoning alongside its actions, and render every row with the chat template the agent uses at serving time. What it costs in general ability The student got worse at general instruction following. On IFEval, prompt-strict fell from 0.819 to 0.741 and instruction-strict from 0.872 to 0.813, a drop of roughly 6 to 8 points. AWS calls this "the expected cost of specialization" and notes that a much larger drop would be a warning that distillation narrowed the model too far. If you plan to use the model for other jobs as well, test for this regression. Serving it The distilled model does not run tools on its own. A small deterministic harness prompts the model, parses the tool call, runs the tool, appends the observation and loops until a final answer. AWS serves the LoRA student as an adapter on top of the base model, as two inference components on one real-time endpoint using the DJL LMI container version 0.36.0, with vLLM bundled . The base weights load once and both components share them, so the adapter costs adapter-sized memory instead of a second full copy. The endpoint can run on an ml.g5.2xlarge, ml.g6.2xlarge or ml.g6e.2xlarge instance, in that order of preference. AWS also proposes a cheap pre-check before the full evaluation: score one generated step at a time against the teacher's action, grading tool selection only. If the student does not beat the base model there, fix training first. What the post does not prove - No cost or latency numbers. AWS says it does not publish a per-task cost or latency figure, because it depends on traffic, sequence lengths and concurrency. The pitch that you trade a variable per-token bill for a fixed hourly instance cost is logical, but you must measure it. A hosted endpoint bills by the hour whether it serves one task or thousands, so low-traffic workloads may not save money. - One task family. A shopping benchmark with a clean programmatic grader is close to the best case for distillation. Agents without an automatic scorer need another way to filter trajectories. - Teacher size. GLM-5.3 is described as a mixture-of-experts model with roughly 40 billion active parameters, about ten times the student's 4 billion. The student recovers its quality because the task fits a 4B model. AWS warns that across too large a gap "the student copies the teacher's format without learning to solve," with trajectories that lengthen while accuracy stays flat. - The hardest cases. SeqKD lifts easy and mid-difficulty problems but plateaus on the hardest ones, where the student must recover from its own mistakes. AWS suggests an on-policy stage afterwards. - Terms of use. Training on a frontier model's outputs can break a provider's terms. Check the license of the teacher before you distil from it. We have covered the disputes in Anthropic's accusations over distillation by fake accounts https://explainx.ai/blog/anthropic-alibaba-claude-distillation-25000-fake-accounts-2026 and in our look at black-box distillation methods https://explainx.ai/blog/proxy-kd-black-box-llm-distillation-fable-5-2026 . Related evidence that small models can win on narrow work This result fits a pattern. Coinbase reported that a fine-tuned Qwen3.5 9B beat frontier models on fraud detection, which we covered in Coinbase's Qwen fraud agent https://explainx.ai/blog/coinbase-qwen3-5-9b-fraud-agent-beats-opus-4-5-newer-models-regress-owning-intelligence-2026 . AWS has also shown reinforcement learning lifting gpt-oss-20b on Wordle for about $339, covered in our SageMaker multi-turn RL walkthrough https://explainx.ai/blog/amazon-sagemaker-multi-turn-reinforcement-learning-wordle-gpt-oss-20b-agentcore-339-dollars-2026 . Together they suggest a recipe: use a frontier model to explore and generate data, and a small model to run the repeated work. Should you try it? Try it if you have an agent that runs the same kind of task thousands of times, a grader that can tell solved from failed, and a teacher whose terms allow it. Start small: sample a few hundred trajectories, filter, fine-tune a 4B to 9B model with LoRA, and compare it with the base model and the teacher on held-out problems. Count tokens and GPU hours for both paths before you decide. Skip it if your tasks are open-ended, vary a lot, or need broad general ability. In those cases the 6 to 8 point drop on IFEval is a sign of what you give up. The recipe is sound, the numbers are promising, and the cost story is still yours to measure.