Build agentic retrieval on a Managed Amazon Bedrock Knowledge Base with CDK

wpnews.pro

Artificial Intelligence #

This post builds enterprise agentic retrieval on a Managed Amazon Bedrock Knowledge Base with the AWS Cloud Development Kit (AWS CDK). To see why that’s worth building, it helps to start with where ordinary Retrieval Augmented Generation (RAG) breaks down. Most RAG systems start the same way: they embed the question, fetch the nearest chunks from one index, hand them to a model, and return the answer. One retrieval, one generation. For a large class of questions that’s the right design, and there’s no reason to build anything more complicated.

It fails in a recognizable way. Ask a question whose answer spans two corpora, and a single retrieval returns half of what the model needs, which is worse than returning nothing, because the answer comes back fluent and incomplete. Ask a question where the right source depends on the question, and a single index has no mechanism to choose. Ask a question that needs a second look at the corpus after seeing the first result, and there is nowhere for that second look to happen.

Enterprise agentic retrieval answers all three by putting a reasoning agent in the loop. The agent decides whether to retrieve, which knowledge base to consult, and whether to go back for more before it composes an answer. This is the direction Amazon Bedrock Knowledge Bases have evolved. With the launch of the Managed Knowledge Base, retrieval is no longer a single embed-and-fetch call. It has become an agentic capability: multi-turn planning, tool-driven retrieval, and grounded, cited answers. The AgenticRetrieveStream API offers this as a first-class service feature rather than something you assemble yourself. That works, and it also introduces a genuinely harder operational problem. A single-shot pipeline is a function you can reason about by reading it. An agent that reasons and retrieves in a loop takes a different path on every question. By default you can see neither the path it took nor whether the answer it produced was any good. Observability stops being a nice-to-have and becomes the only way to know the system works.

In August 2024, Build an end-to-end RAG solution using Knowledge Bases for Amazon Bedrock and the AWS CDK deployed the single-shot design as a CDK application: an Amazon Simple Storage Service (Amazon S3) corpus, an Amazon OpenSearch Serverless collection and index, a knowledge base, and a data source, provisioned as CDK stacks whose order you sequenced by hand. Three things have changed since it was published.

Amazon Bedrock now offers a managed knowledge base (Type: MANAGED), which owns the ingestion, storage, indexing, and retrieval itself. The OpenSearch Serverless collection and index that post provisions are no longer something you need to create, scale, or pay for with OpenSearch Compute Units (OCUs).
Amazon Bedrock AgentCore provides the agentic half , a Gateway that exposes each knowledge base as a Model Context Protocol (MCP) tool, a Runtime that hosts the reasoning agent, and AgenticRetrieveStream, which reranks internally and returns a synthesized, cited answer. Agentic retrieval is availableonly on the Managed Knowledge Base.
The observability and evaluation story is now something you can provision , not something you bolt on. This solution ships twoAmazon CloudWatch dashboards spanning seven layers of telemetry, plus two forms of evaluation, on-demand and continuous, from the same deployment.

This post builds that system: an agent that routes across two knowledge bases, retrieves through the Gateway, and returns a grounded, cited answer, with every layer of its behavior instrumented, as a single AWS CDK application in Python. One command deploys all of it, and CDK derives the order.

If you read the 2024 post, the path from an Amazon S3 bucket to the knowledge base will look familiar, and everything after it is new: the vector store is gone, the retrieval is agentic, and roughly half of what the app provisions exists to tell you what the agent did. If you didn’t, this post stands on its own. The services at play and the path one question takes through them: the agent decides whether to retrieve and which knowledge base to consult, the Gateway exposes each knowledge base as a tool it can call, the Managed Knowledge Bases search and cite, and the loop repeats as many times as the question needs. Everything the agent does along the way lands in Amazon CloudWatch.

Background #

Before the walkthrough, it helps to establish three things: what makes retrieval agentic, why the Managed Knowledge Base is the right foundation for it, and why observability and evaluation belong in the design rather than bolted on later.

From RAG to enterprise agentic retrieval

Classic Retrieval Augmented Generation (RAG) does one retrieval and one generation. Enterprise agentic retrieval puts a reasoning agent in the loop: it decides whether and what to retrieve, can retrieve several times to refine, chooses which knowledge base is relevant (semantic routing), and only then composes a grounded answer with citations. With the Managed Knowledge Base, this is a first-class service capability, the AgenticRetrieveStream API performs the multi-turn planning, executes the retrieval, and returns a cited answer, rather than an orchestration layer you build and maintain over a raw vector search. That produces better answers on complex questions, but it also produces a more complex system to operate, which is exactly why observability and evaluation are built in from the start.

Managed compared to do-it-yourself Knowledge Bases

Amazon Bedrock now offers a Managed Knowledge Base (Type: MANAGED): Amazon Bedrock manages the ingestion, storage, indexing, and retrieval for you, including embedding and reranking with service-managed models by default, so there is no vector database to provision, scale, or patch. This differs from a Customer-managed (DIY) Knowledge Base, where you choose and operate the vector store (for example, Amazon OpenSearch Serverless) yourself.

Two differences matter most for this solution:

Because agentic retrieval and the AgentCore Gateway connector are available only on the managed knowledge base, it is the natural foundation for enterprise agentic retrieval. It also means Amazon Bedrock manages the storage and indexing for you, so there is no OpenSearch Serverless collection or OCU capacity to provision in this design. The customer-managed option remains the right choice when you want to select and operate your own vector store, for example to reuse an existing OpenSearch Serverless collection or to meet specific data-residency, tuning, or compliance requirements. For enterprise agentic retrieval, the managed knowledge base is what makes the capability possible.

Why observability and evaluation

An agentic system that returns an answer isn’t enough for production. You need to see how it behaves (latency, call volume, token usage), how good the retrieval and answers are, and you need those signals continuously. This solution ships two CloudWatch dashboards spanning seven layers of telemetry, plus two forms of evaluation (on-demand and continuous), all provisioned by the same CDK application. The grounding, citations, and evaluation layers also support responsible use. Cited answers trace back to their source passages so responses stay auditable, and the faithfulness and correctness scores let you catch answers drifting from their sources rather than finding out in production.

Why CDK for this particular solution

Any infrastructure-as-code tool can create these resources. CDK earns its place here for a reason specific to this architecture: the four stacks are densely cross-referenced. The Gateway needs both knowledge base ids. The agent needs the URL and ARN for the Gateway. The dashboards need the KB ids, the Gateway ARN, and the agent runtime id, seven values crossing stack boundaries, each one a dimension on a metric or a target on a connector.

Written as raw templates, that is a set of stack exports and cross-stack lookups kept in agreement by hand, plus a script that deploys in the right order and threads outputs forward, and both drift the moment anything is renamed. In CDK you pass the construct, and the export and the dependency edge are derived from that one act of passing it. The 2024 solution had three stacks and a mostly linear dependency chain. At four stacks with seven values crossing between them, the difference stops being stylistic.

Solution overview #

Here is the path a single question takes through the system, end to end. It’s worth a slow look, because the rest of the post walks through it left to right.

The following diagram traces the end-to-end agentic retrieval workflow. A user’s question enters the agent on the AgentCore Runtime, which routes it through the AgentCore Gateway to one of two AgenticRetrieveStream tools, one per managed knowledge base, financial and weather, each ingested from Amazon S3. The selected tool returns a grounded, cited answer, and every step streams to Amazon CloudWatch and AWS X-Ray. The numbered badges map to the following walkthrough.

The numbered steps in the diagram map to the workflow, which is as follows.

Step 0 — one-time, at deploy. Before any question is asked, each corpus in Amazon S3 is ingested into its own Managed Knowledge Base: Amazon Bedrock parses, chunks, embeds, and indexes it automatically. This happens once when the stacks deploy, it isn’t part of the per-question path, so it is drawn as a dashed arrow and marked step 0. Every question then follows steps 1–6:

A user sends a question to the agent hosted on the Amazon Bedrock AgentCore Runtime. The Runtime auto-instruments the whole reason-and-act loop with OpenTelemetry (OTEL) spans, so the system is observable from the very first call.
The agent’s reasoning model plans and routes across knowledge bases. Because each knowledge base is exposed as its own retrieval tool, the agent selects the tool whose topic matches the question, financial or weather, rather than searching one blended index.
The selected tool call is brokered by the AgentCore Gateway over MCP , authorized by the agent’s IAM task role and signed with SigV4, which invokes that knowledge base’s AgenticRetrieveStream API on the Managed Knowledge Base.
AgenticRetrieveStream does the within-knowledge-base work: it decomposes the question into sub-queries, retrieves iteratively from the data store that Amazon Bedrock manages (ingested from the corpora in Amazon S3), and synthesizes a grounded, cited answer that streams back through the Gateway.
The agent checks whether the returned context is sufficient. If not, it loops, routing and retrieving again, before composing its final answer. If so, it returns the cited answer to the user.
Throughout, the Runtime emits spans, token counts, and metrics to Amazon CloudWatch and AWS X-Ray, populating the seven observability layers and feeding both the on-demand and continuous evaluation scores.

Internally, that workflow is one CDK app, app.py, that declares four stacks. You don’t choose the order, CDK derives it from the construct references you pass:

That is the whole wiring. Those four constructor calls are what replace the export/import bookkeeping and the deploy script.

The agent runs a reason-and-act loop: it thinks (a large language model (LLM) call), decides which KB tool to call, reads what came back through the Gateway, and often retrieves again before composing its final, cited answer. Every step is auto-instrumented by the Runtime, so the seven observability layers fill from real traffic.

The seven layers. L1 KB-native metrics, L2 ingestion, L3 agentic retrieval quality, L4 Gateway/MCP metrics, L5 OTEL span tree , L6 token usage, and L7 evaluation scores. Layers 1, 4, and 5 are emitted automatically. Layers 3, 6, and 7 are published as custom metrics by the driver notebook.

Everything tunable lives in one file, config.py:

Changing PROJECT_NAME gives you an independently-named copy of the whole solution, so two deployments can coexist in one account. The driver notebook reads the same constant, so it follows the rename automatically, there are no stack names to keep in sync by hand.

Describe the dataset #

The solution ships with two small corpora, bundled in the repository under data/:

Financial — A synthetic Octank Financial 10-K (octank_financial_10K.pdf, ~198 KB).
Weather — Areal, publicly available U.S. Congressional Research Service report on tornadoes (IF12695 , tornadoes_report.pdf, ~560

source & further reading

aws.amazon.com — original article Groundhog Day for AI agents: Prompt caching patterns for using OpenAI models on Amazon Bedrock AWS Weekly Roundup: Amazon Polly Generative engine, TorchNeuron for Trainium, and more (October 26, 2026) Infinity: A runtime for extreme agent concurrency