{"slug": "aws-builds-a-multi-agent-part-substitution-assistant-with-nova-embeddings-and-it", "title": "AWS Builds a Multi-Agent Part-Substitution Assistant With Nova Embeddings and AgentCore: What It Shows", "summary": "AWS published a reference build on November 2, 2026 for a multi-agent part-substitution assistant that splits work across three Claude model tiers — Claude Haiku 5.5 for query routing, Claude Sonnet 5.5 for running Catalog, Evaluation and Knowledge specialist tools, and Claude Opus 5.5 for synthesis only when two or more specialists contribute — with deterministic Python tolerance-rule code making the spec-fit verdicts and the engineer retaining every substitution decision. The AWS Machine Learning blog post, which ships with five Jupyter notebooks and runs on synthetic aerospace data, claims to compress part evaluation \"from days to minutes\" but reports no timing study on real engineers. AWS says skipping the Opus supervisor for single-specialist answers avoids about 8 seconds of latency with no quality gain, and each evaluation writes an audit trail entry aligned with AS9100 sections 8.4 and 8.5.6.", "body_md": "AWS has published a reference build for an AI assistant that helps engineers evaluate replacement parts, and the useful part is not the aerospace setting. It is a clean example of how to split an agent system by role: a small model routes, a mid-size model runs tools, a large model synthesizes, and plain code makes the decision that must be reproducible. The [AWS Machine Learning blog post](https://aws.amazon.com/blogs/machine-learning/agentic-engineering-intelligence-with-amazon-nova-multimodal-embeddings-and-amazon-bedrock-agentcore/?ref=explainx), published November 2, 2026, comes with five Jupyter notebooks.\n\nIt is a vendor tutorial run on synthetic data. We summarize what it shows, which numbers are measured and what to treat as a design pattern rather than a proven result.\n\n## The problem\n\nWhen a qualified aerospace component goes obsolete or hits a long lead time, an engineer spends days cross-referencing specifications across vendor datasheets, interface control documents (ICDs) and qualification records, then writes a substitution justification for review. AWS notes that a wrong substitution on a flight-critical part can cause mission failure, and that the evidence trail is often spread across email and institutional memory.\n\nAWS's goal is to cut the manual cross-referencing. Its headline claim is compressing the evaluation \"from days to minutes.\" That is stated in the description, and the post reports no timing study on real engineers, so read it as an aim.\n\n## The design rule: the engineer decides\n\nThe assistant is built around one rule. The engineer makes every decision, and the system assembles the evidence. That rule decides where models are used and where deterministic code is used. AWS names three pillars:\n\n1. **Deterministic spec-fit evaluation.** Python tolerance-rule code scores each candidate against the reference part and the engineer's requirements. The same inputs give the same pass or flag verdicts, so results can go into a formal review.\n2. **Multi-agent orchestration.** A lightweight planner routes each query to one or more of three specialists (Catalog, Evaluation, Knowledge). A supervisor synthesizes their findings only when more than one contributed.\n3. **Human-in-the-loop decision.** The system drafts justification memos and Integration Impact Assessments but does not recommend or approve a substitution. Each evaluation writes an audit trail entry aligned with AS9100 sections 8.4 and 8.5.6.\n\n## The architecture, piece by piece\n\n### Three model tiers\n\n| Role | Model | Job | \n|---|---|---|\n| Planner | Claude Haiku 5.5 | Classifies the query and returns a numbered routing plan in a couple of seconds | \n| Specialists | Claude Sonnet 5.5 | Run the tools for Catalog, Evaluation and Knowledge | \n| Supervisor | Claude Opus 5.5 | Synthesizes results, only when two or more specialists ran | \n\nAWS explains the split: routing is classification with no tool calls, so it does not need a large reasoning model. The supervisor is skipped for single-specialist answers because it adds about 8 seconds of latency with no quality gain. For pricing and behavior of the small model, see our [Claude Haiku 5.5 launch coverage](https://explainx.ai/blog/claude-haiku-5-5-launch-pricing-benchmarks-sonnet-cache-cut-2026). For a similar planner, worker and critic split from the research side, see our write-up of an [agent harness with a DAG planner](https://explainx.ai/blog/agent-harness-dag-planner-worker-critic-budget-pressure-2026).\n\n### Scoped tools per specialist\n\nEach specialist is a [Strands](https://strandsagents.com/?ref=explainx) agent with a narrow prompt and only the tools it needs. The Knowledge specialist cannot call catalog discovery tools, and the Catalog specialist cannot call evaluation tools. This keeps behavior predictable and limits what a confused model can do. [AWS](https://github.com/aws-samples/amazon-nova-samples/tree/main/multimodal-embeddings/repeatable-patterns/agentic-engineering-intelligence-nova-mme-agentcore?ref=explainx) also treats the structured vendor field as authoritative and forbids specialists from guessing a manufacturer from an ID, file name, photo or web result, because folder IDs repeat across datasets.\n\n### Deterministic scoring\n\nThe scoring function runs as ordinary Python against a hand-written requirements schema. Each component category lists critical specs and rules, such as a voltage that must match or a mass that must be within 20 percent of the reference. The tool returns per-spec pass, flag or review results, counts, out-of-tolerance items and lifecycle flags, plus a disclaimer: \"Evidence only. This tool does not recommend or approve a substitution; the engineer decides.\"\n\nThe specialist can present and explain this table but cannot change a verdict. Logging follows the same principle: the orchestration layer reads the tool-call history and writes the audit entry itself, so it does not depend on the model remembering to do it.\n\n### Nova Multimodal Embeddings in two roles\n\n- **Retrieval.** Datasheets, ICDs and performance charts are embedded so text and visuals share one semantic space. An engineer can type \"torque curve for the 0.06 Nms wheel\" or upload a chart and get the matching diagram with a source.\n- **Visual similarity.** Two part photos are embedded and compared by cosine similarity, and Claude then describes the physical differences, such as mounting pattern or housing size. AWS reports about 0.84 for two wheels of the same product family and 0.40 for wheels of different classes. These are examples from its own data.\n\nThere are two retrieval paths. Text and document queries use a managed knowledge base routed through an AgentCore Gateway. Image-as-query uses a native Amazon S3 Vectors index at 1,024 dimensions, because AWS says the managed knowledge base retrieval API does not accept image input. Results are deduplicated to the best vector per part, since each part contributes several vectors.\n\n## What was measured\n\nAgentCore online evaluation publishes scores to CloudWatch. The test set holds 36 queries in six groups: catalog discovery (8), knowledge and ICD retrieval (8), multi-specialist chains (6), deterministic evaluation (5), text-to-visual retrieval (5) and safety refusals (4), where the agent must refuse to recommend or approve.\n\nAWS reports, from a snapshot on the synthetic dataset (30-day averages):\n\n| Level | Evaluator | Score | \n|---|---|---|\n| Session | Goal Success Rate | 0.892 | \n| Trace | Correctness | 0.968 | \n| Trace | Helpfulness | 0.956 | \n| Trace | Tool Parameter Accuracy | 0.980 | \n\nAWS adds that all trace-level quality metrics were at or above 0.95. It explains the lower Goal Success Rate by the test design: the four safety prompts ask the agent to approve a substitution, the agent correctly refuses, and the evaluator scores a correct refusal as an unmet goal. Tool Selection Accuracy rose from 0.809 to 0.861 between evaluation runs, which it attributes to three changes: consolidating to a single managed knowledge base, fixing a missing permission on the image-as-query path that had caused a silent fallback to text search, and tightening the planner's routing to avoid chaining unnecessary specialists.\n\nThat second fix is the most instructive. A permission error made the agent quietly fall back to a worse retrieval path, and only an evaluation caught it. If you build agents that call tools, test each path separately and log which one served each answer.\n\n## What the results do not show\n\n- **Synthetic data.** The corpus is seven reaction wheels from a fictional manufacturer, with momentum storage from 0.003 to 5.0 Nms. Real catalogs are messier, with conflicting datasheets and scanned documents. AWS itself says to expect different numbers on your own runs.\n- **LLM-graded metrics.** Correctness and helpfulness are scored by evaluators that are themselves model-based, and the post gives limited detail on the grading. Treat them as trend indicators, not ground truth.\n- **Small test set.** 36 queries, four of them refusals. A difference of 0.05 on a set this size is within noise.\n- **No time study.** \"Days to minutes\" is not measured on real engineers.\n- **No cost or latency totals.** The post gives the 8-second supervisor figure and a planner time of \"a couple of seconds,\" but no end-to-end cost per query.\n- **Vendor setting.** The stack is AWS-only, using Bedrock, AgentCore, S3 Vectors and Claude models served on Bedrock.\n\n## Lessons you can reuse\n\n1. **Put the model where judgment is fuzzy and code where it must be reproducible.** The spec-fit verdict is code. Explanation and retrieval are models.\n2. **Use small models for routing.** Classification does not need your biggest model.\n3. **Skip synthesis when one specialist answered.** It saves latency.\n4. **Scope tools per agent.** Fewer tools means fewer wrong calls.\n5. **Make the system write its own audit log.** Do not ask the model to remember.\n6. **Evaluate each retrieval path.** Silent fallbacks hide bugs.\n7. **Keep the human in the decision loop** wherever a mistake is expensive.\n\n## How it connects to other agent work\n\nThe pattern echoes a trend we have followed: small specialized components around a few expensive calls. See [Strands Decider](https://explainx.ai/blog/strands-decider-2b-open-source-decision-model-2026), a 2B open decision model from the same SDK ecosystem, and our look at [AgentCore Payments](https://explainx.ai/blog/aws-agentcore-payments-anchor-browser-x402-agents-pay-without-private-keys-2026), where agents act with constrained permissions. If you want a way to cut per-step cost further, our write-up of [AWS's SeqKD distillation walkthrough](https://explainx.ai/blog/aws-sagemaker-seqkd-4b-shopping-agent-matches-glm-5-3-teacher-0-541-vs-0-540-distillation-guide-2026) shows how to train a small model on a larger one's trajectories.\n\nAgents that touch real systems also need guardrails. [AgentBeam](https://agentbeam.com/?ref=explainx), the agent security platform from the explainx.ai team, stops AI agents before they take dangerous actions, which complements the design here, where the agents are barred by construction from approving anything.\n\n## Should you try it?\n\nYes, if you work with technical documents full of charts and tables and you need traceable outputs. Clone the notebooks, replace the synthetic parts with a slice of your own corpus and measure retrieval and tool selection before you trust any single score. Keep the decision step in code and the sign-off with a person. No, if you expect the posted numbers to carry over to production: they come from fictional parts and a 36-query test, and the real test is a messy catalog and an engineer who has to sign.", "url": "https://wpnews.pro/news/aws-builds-a-multi-agent-part-substitution-assistant-with-nova-embeddings-and-it", "canonical_source": "https://explainx.ai/blog/aws-nova-multimodal-embeddings-agentcore-strands-aerospace-part-substitution-agent-36-query-eval-guide-2026?utm_source=rss&utm_medium=feed&utm_campaign=blog", "published_at": "2026-11-02 00:00:00+00:00", "updated_at": "2026-11-02 17:53:19.798009+00:00", "lang": "en", "topics": ["ai-agents", "artificial-intelligence", "large-language-models", "ai-tools"], "entities": ["AWS", "Amazon Nova", "Amazon Bedrock AgentCore", "Claude Haiku 5.5", "Claude Sonnet 5.5", "Claude Opus 5.5", "Strands", "AS9100"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/aws-builds-a-multi-agent-part-substitution-assistant-with-nova-embeddings-and-it", "markdown": "https://wpnews.pro/news/aws-builds-a-multi-agent-part-substitution-assistant-with-nova-embeddings-and-it.md", "text": "https://wpnews.pro/news/aws-builds-a-multi-agent-part-substitution-assistant-with-nova-embeddings-and-it.txt", "jsonld": "https://wpnews.pro/news/aws-builds-a-multi-agent-part-substitution-assistant-with-nova-embeddings-and-it.jsonld"}}