Agentic engineering intelligence with Amazon Nova Multimodal Embeddings and Amazon Bedrock AgentCore

AWS published a technical walkthrough for building a multi-agent engineering assistant on Amazon Bedrock AgentCore that uses Amazon Nova Multimodal Embeddings and the open source Strands Agents SDK to assemble part-substitution evidence for aerospace components. The system routes queries through a planner (Claude Haiku 5.5) to three specialist agents (Claude Sonnet 5.5) covering catalog search, deterministic Python tolerance-rule scoring, and lifecycle knowledge, with a supervisor (Claude Opus 5.5) synthesizing results when more than one specialist runs. AWS states the engineer remains the decision-maker: the system drafts justification memos and Integration Impact Assessments and writes audit trail entries aligned with AS9100 sections 8.4 and 8.5.6, but never approves a substitution itself.

Artificial Intelligence https://aws.amazon.com/blogs/machine-learning/ Agentic engineering intelligence with Amazon Nova Multimodal Embeddings and Amazon Bedrock AgentCore When a qualified aerospace component goes obsolete or hits a long lead time, an engineer spends days cross-referencing specifications across vendor datasheets, interface control documents ICDs , and qualification records before documenting a substitution justification for review. An incorrect substitution on a flight-critical component can cause mission failure, yet the evidence trail behind each decision is often spread across email and undocumented institutional knowledge, so there is no formal, repeatable package for a reviewer to check. A previous post, Manufacturing intelligence with Amazon Nova Multimodal Embeddings https://aws.amazon.com/blogs/machine-learning/manufacturing-intelligence-with-amazon-nova-multimodal-embeddings/ , showed how the model retrieves answers locked inside engineering diagrams, thermal plots, and datasheet charts that text-only pipelines miss. This post picks up where that retrieval work left off, putting those capabilities inside a multi-agent system that assembles substitution evidence, from retrieval to a drafted assessment. In this post, we show how to build a multi-agent engineering assistant that cuts the manual cross-referencing behind part substitution evaluation using Amazon Bedrock AgentCore https://aws.amazon.com/bedrock/agentcore/ , Amazon Nova Multimodal Embeddings https://docs.aws.amazon.com/nova/latest/nova2-userguide/embeddings.html , and the open source Strands Agents SDK https://strandsagents.com/ . A planner routes work across three specialist agents that search a multimodal knowledge base, score candidates against deterministic tolerance rules, compare part photos, and ground lifecycle claims in live web data. The engineer remains the decision-maker throughout, because the system assembles evidence and drafts documents without ever approving a substitution on its own. Solution overview The assistant is built around one design rule, which is that the engineer makes every decision and the system assembles the evidence for it. That rule determines where we use large language models LLMs and where we use deterministic code instead. Three pillars define the architecture: - Deterministic spec-fit evaluation – Tolerance-rule code written in Python scores each candidate part against the reference part and the engineer’s stated requirements. Because the same inputs consistently produce the same pass/flag verdicts, the results can go into a formal engineering review. - Multi-agent orchestration – A lightweight planner routes each query to one or more of three specialist agents Catalog, Evaluation, Knowledge , and a supervisor synthesizes their findings only when more than one specialist contributed. Complex questions such as “Check if this wheel is going end-of-life, find alternatives, and score them” chain all three specialists in sequence. - Human-in-the-loop decision – The system drafts justification memos and Integration Impact Assessments for engineering review, but it doesn’t recommend or approve a substitution. Every evaluation also writes an audit trail entry aligned with AS9100 sections 8.4 and 8.5.6. The following diagram shows the architecture. Amazon Bedrock AgentCore is a platform to build, connect, and optimize agents at scale, with any framework or model. In this solution, chat requests from the engineer’s application flow into a multi-agent graph hosted on AgentCore runtime: a planner Claude Haiku 5.5 produces a routing plan, specialists Claude Sonnet 5.5 execute their tools, and a supervisor Claude Opus 5.5 synthesizes their results when more than one specialist ran. The specialists retrieve from a managed knowledge base that we created with Amazon Bedrock Managed Knowledge Bases https://docs.aws.amazon.com/bedrock-agentcore/latest/devguide/gateway-target-connector-managed-kb.html using Amazon Nova Multimodal Embeddings, routed through an AgentCore Gateway, a capability of Amazon Bedrock AgentCore, alongside the Web Search connector for live web data. For image-as-query uploading a chart to identify a part , the agent calls a native Amazon S3 Vectors index directly, because the managed knowledge base retrieval API doesn’t accept image input. Amazon S3 Vectors is a capability of Amazon Simple Storage Service Amazon S3 . A separate fixed chain Knowledge, then Evaluation, then Supervisor produces the formal Integration Impact Assessment IIA document. Amazon Nova Multimodal Embeddings appears in two distinct roles: 1. Multimodal retrieval – Datasheets, ICDs, and performance charts are embedded with Nova Multimodal Embeddings so text and visual content share one semantic space. An engineer can enter “torque curve for the 0.06 Nms wheel” or upload a chart image, and retrieval returns the matching diagram with source attribution. 2. Visual similarity scoring – When two candidate parts are compared, Nova Multimodal Embeddings produces a quantitative cosine similarity between their photos, and Claude analyzes the specific physical differences mounting pattern, connector placement, housing geometry . The score adds supporting evidence to the deterministic spec comparison, which is what the fit verdict is based on. Prerequisites To follow along, you need: - An AWS account with access to Amazon Bedrock in the US East N. Virginia AWS Region. - Model access to Amazon Nova Multimodal Embeddings, Claude Haiku 5.5, Claude Sonnet 5.5, and Claude Opus 5.5 in Amazon Bedrock. For model availability by Region, see Model support by AWS Region in Amazon Bedrock https://docs.aws.amazon.com/bedrock/latest/userguide/models-regions.html . - A managed knowledge base created with Amazon Bedrock Managed Knowledge Bases, using Amazon Nova Multimodal Embeddings over your parts corpus datasheets, ICDs, descriptions for text and document retrieval. - An Amazon S3 Vectors index over your part photos and datasheet chart crops, for the image-as-query path. - An Amazon Bedrock AgentCore Gateway with the Web Search connector and a managed knowledge base target. - Python 3.10 or later and Jupyter for running the notebooks. - Docker, if you adapt the deployment recipe in notebook 03 to host the agent on AgentCore runtime. - The sample code from the GitHub repository https://github.com/aws-samples/amazon-nova-samples/tree/main/multimodal-embeddings/repeatable-patterns/agentic-engineering-intelligence-nova-mme-agentcore . Solution walkthrough The following sections describe each part of the system in the order a query uses it. We start with the parts corpus and the specialist agents, then cover routing, deterministic scoring, the two ways the assistant uses Nova Multimodal Embeddings, the fixed chain that produces formal documents, and deployment to AgentCore runtime. The complete code sample is a set of five Jupyter notebooks in the GitHub repository https://github.com/aws-samples/amazon-nova-samples/tree/main/multimodal-embeddings/repeatable-patterns/agentic-engineering-intelligence-nova-mme-agentcore , and each notebook runs on its own. The notebooks import from mei core , a self-contained copy of the agent and tool modules in the same folder, and they include their own parts data, chart crops, and evaluation query set. You can run every snippet in this post from the notebooks without other application code. Structure the parts corpus The sample dataset is seven reaction wheels from a fictional manufacturer AnyCompany spanning 0.003–5.0 Nms of momentum storage. Each part is a folder that pairs structured data with the source documents an engineer actually reads, and this mix is what makes the corpus multimodal. The specs.json file holds the machine-readable fields that drive deterministic scoring, while the ICD, product photo, and rendered datasheet page carry the visual and narrative detail that structured fields leave out. The specs.json file captures both the engineering specifications and a lifecycle profile, which is what lets the assistant flag obsolescence and lead-time risk alongside spec fit. The vendor field is stated explicitly for every part. Because catalog folder IDs and file names are reused across datasets, every specialist prompt treats the structured vendor value returned by get part specs as authoritative and is forbidden from guessing a manufacturer from an ID, a file name, a photo, or a web search result. Taking the manufacturer from structured data keeps the assistant from attributing a part to the wrong supplier because the model thinks it recognizes the hardware. Each part folder also carries two assets that text-only indexing cannot use. The product photo feeds visual similarity scoring, and the rendered datasheet page carries torque-box and torque-curve plots that answer performance questions no OCR text layer captures. Amazon Nova Multimodal Embeddings embeds both into the same vector space as the text, so a query can match a spec value, a paragraph, or a plot. The following sample shows four datasheet pages from the corpus, spanning three orders of magnitude of momentum storage. The template repeats across the family, but the product photo and the pair of performance plots change with every part. These Regions show the housing geometry, mounting pattern, torque envelope, and power draw, which the spec fields don’t state directly, so we embed them alongside the text. Define specialists with scoped tools Each specialist is a Strands agent with a narrow prompt and only the tools it needs. Scoping tools per specialist keeps behavior predictable, because the Knowledge specialist has no access to the catalog discovery tools and the Catalog specialist has no access to the evaluation tools. The following code creates the Evaluation specialist. The following diagram shows how the planner routes a query to the specialists and when the supervisor synthesizes their results. The list after it describes what each specialist does. - Catalog – Discovers parts through knowledge base search and exhaustive spec filtering kb search , filter parts by spec , get part specs , get substitution requirements . - Evaluation – Scores candidates deterministically and compares parts visually score substitution fit , compare parts , visual similarity score , check qualification status , generate substitution memo . - Knowledge – Answers questions from datasheets and ICDs, checks live end-of-life status, and retrieves visual content get part specs , web search eol , retrieve visual content . Use a small model for routing and a large model for synthesis Routing is a classification task with no tool calls, so it doesn’t need a large reasoning model. A Haiku-based planner returns a numbered routing plan in a couple of seconds. The Opus supervisor runs only when two or more specialists contributed, because synthesizing a single specialist’s already-complete answer adds about 8 seconds of latency with no quality gain. Splitting the work across three model tiers matches cost and latency to each stage. Haiku classifies the query, Sonnet runs the tools, and Opus writes the combined answer, which is the step where response quality matters most. Keep the scoring deterministic The spec-fit evaluation runs as ordinary Python code against a hand-written requirements schema. The schema defines the critical specs and tolerance rules for each component category, and score substitution fit compares each spec, counts passes and flags, and surfaces lifecycle risk. Because the verdict must be reproducible for an engineering review package, no LLM is involved in this step. The tool returns a structured verdict with an explicit disclaimer: The specialist presents this table and explains the flags, but it can’t change a verdict. The audit trail follows the same division of work. After every evaluation, the orchestration layer reads the specialist’s tool-call history and writes the audit entry itself, so logging doesn’t depend on the model remembering to do it. Score visual similarity with Amazon Nova Multimodal Embeddings Form factor matters when a substitute must fit an existing bracket. The visual similarity score tool embeds both part photos with Nova Multimodal Embeddings and computes cosine similarity, then asks Claude to describe the specific observable differences.