When I started designing the RAG architecture for the internal AI assistant project, one of the first decisions I had to make was:
Where should the application's knowledge live, and which component should be responsible for turning that knowledge into something the model can retrieve?
There are several ways to build a RAG system on AWS.
I could assemble and manage the retrieval components myself, use a vector database directly, or use Amazon Bedrock Knowledge Bases to manage much of the RAG data pipeline.
During the project, I explored the architecture from both perspectives before settling on an Amazon Bedrock Managed Knowledge Base.
This article explains that decision and the technical tradeoffs behind it.
The application needed to answer questions using internal company documents.
The source documents included:
Brand Guidelines
Script Bank
Caption Bank
Client Proposal Template
My initial mental model was:
Documents
↓
Amazon S3
↓
Vector Storage
↓
Similarity Search
↓
Foundation Model
↓
Response
But implementing that architecture means taking responsibility for considerably more than vector storage.
The complete pipeline involves:
Document ingestion
↓
Document parsing
↓
Chunking
↓
Embedding generation
↓
Vector indexing
↓
Metadata
↓
Similarity search
↓
Retrieval
↓
Context construction
↓
Model invocation
↓
Source attribution
At that point, the most important architectural question became:
Which parts of the RAG pipeline should my application own, and which parts should I delegate to a managed AWS service?
That question led me toward Amazon Bedrock Managed Knowledge Bases.
Amazon Bedrock Knowledge Bases provides a managed abstraction for building RAG applications.
Instead of my application owning every component of the retrieval pipeline, the architecture becomes:
Company Documents
│
▼
Amazon S3
│
▼
┌───────────────────────┐
│ Amazon Bedrock │
│ Managed Knowledge │
│ Base │
│ │
│ Parsing │
│ Chunking │
│ Embeddings │
│ Indexing │
│ Retrieval │
└───────────┬───────────┘
│
Relevant Context
│
▼
Application Layer
│
▼
Foundation Model
│
▼
Response
That creates a significantly different responsibility boundary.
My application remains responsible for the business logic, orchestration, model invocation, and user experience.
The Knowledge Base handles much of the document ingestion and retrieval pipeline.
It is important not to think of a managed Knowledge Base as simply "a vector database."
That misses most of the abstraction it provides.
During ingestion, Amazon Bedrock processes the source content, splits it into chunks, generates embeddings, and indexes the resulting vectors for retrieval while maintaining their relationship to the source documents.
Conceptually:
Source Document
│
▼
Parsing
│
▼
Chunking
│
▼
Embeddings
│
▼
Vector Index
│
▼
Retrieval
This means the Knowledge Base owns a substantial portion of the RAG data pipeline that would otherwise have to be implemented and operated by the application.
I wanted the original company documents to remain in an object-storage layer rather than making the vector index the primary source of truth.
Amazon S3 became that source layer.
The document structure was:
ai-creative-content/
├── brand-guidelines/
├── past-scripts/
├── captions/
└── proposals/
This gives the architecture a clear separation:
S3
│
└── Original company documents
Knowledge Base
│
└── Searchable representation of those documents
The S3 data source can be configured with inclusion prefixes, allowing the Knowledge Base to process specific paths within the bucket. It also supports synchronization when documents are added, modified, or deleted.
That separation became particularly useful because the original documents remain easy to manage independently of the retrieval layer.
At the AWS API level, the Knowledge Base data source can be configured with an S3 bucket and optional inclusion prefixes.
For example:
{
"s3Configuration": {
"bucketArn": "arn:aws:s3:::ai-creative-content",
"inclusionPrefixes": [
"brand-guidelines/",
"past-scripts/",
"captions/",
"proposals/"
]
},
"type": "S3"
}
The important part is that the Knowledge Base is not simply scanning an arbitrary bucket.
The data source configuration tells Bedrock which S3 location contains the documents that belong to the Knowledge Base.
The S3 bucket and Knowledge Base also need to satisfy the regional requirements, and the Knowledge Base service role needs permission to access the configured data source.
Up a document to S3 is only the beginning.
The document must be ingested into the Knowledge Base before it can participate in retrieval.
The high-level flow is:
PDF
│
▼
Amazon S3
│
▼
Knowledge Base Data Source
│
▼
Ingestion Job
│
├── Parse
├── Chunk
├── Generate Embeddings
└── Index
│
▼
Queryable Knowledge Base
Ingestion transforms the source content into representations that can be searched semantically.
This distinction is important:
S3 upload
≠
Knowledge Base ingestion
A document existing in S3 does not automatically mean it is available for retrieval.
Once the S3 data source is configured, an ingestion job can be started through the AWS Console or CLI.
aws bedrock-agent start-ingestion-job \
--knowledge-base-id <knowledge-base-id> \
--data-source-id <data-source-id> \
--region us-east-1
The command starts an ingestion job for the specified Knowledge Base and data source.
The response includes an ingestion job ID, which can then be used to monitor the job:
aws bedrock-agent get-ingestion-job \
--knowledge-base-id <knowledge-base-id> \
--data-source-id <data-source-id> \
--ingestion-job-id <ingestion-job-id> \
--region us-east-1
The operational flow is therefore:
StartIngestionJob
│
▼
Ingestion Job
│
▼
Monitor Status
│
▼
COMPLETE
│
▼
Knowledge Base Ready
for Retrieval
This became an important operational distinction during the project.
When troubleshooting RAG, I could no longer assume that:
"The file exists in S3"
meant:
"The Knowledge Base can retrieve it."
The document has to successfully pass through the ingestion pipeline first.
Another useful property of the S3 data source is incremental synchronization.
Suppose the Knowledge Base already contains:
Brand Guidelines
Script Bank
Caption Bank
Proposal Template
Then the caption bank is updated.
The architecture does not require me to manually rebuild the entire retrieval system.
After synchronization, the affected content can be reprocessed:
Existing Knowledge Base
│
Document Change
│
▼
Amazon S3
│
▼
Sync
│
▼
Detect Changed Content
│
▼
Re-process Content
│
┌────────┼────────┐
▼ ▼ ▼
Parsing Chunking Embedding
│
▼
Index
│
▼
Updated Knowledge Base
This is one of the operational advantages of using a managed data source rather than implementing a custom document synchronization pipeline.
Chunking is one of the most important stages of the ingestion process.
A document can contain hundreds or thousands of lines. Treating the entire document as one retrieval unit makes precise retrieval difficult.
Instead, the document is divided into smaller chunks.
Brand Guidelines
│
├── Chunk 1: Company overview
├── Chunk 2: Brand positioning
├── Chunk 3: Brand voice
├── Chunk 4: Social media guidelines
├── Chunk 5: Visual identity
└── ...
Amazon Bedrock supports multiple chunking strategies, including fixed-size, hierarchical, semantic, and no-chunking configurations depending on the Knowledge Base setup.
The resulting chunks are then embedded and indexed for retrieval.
This is an important architectural detail because retrieval quality is influenced not only by the vector store, but also by how the original documents were divided before indexing.
After chunking, each chunk needs a numerical representation that allows semantic similarity to be calculated.
Document Chunk
│
▼
Embedding Model
│
▼
Vector Representation
│
▼
Vector Index
At query time, the user's question is also represented in a form that can be compared against the indexed content:
User Question
│
▼
Query Representation
│
▼
Similarity Search
│
▼
Relevant Chunks
The Knowledge Base uses the embedding configuration selected during its creation to generate the vector representations used for retrieval.
This is another responsibility I did not have to implement as a separate application pipeline.
Once ingestion succeeds, the Knowledge Base can be queried.
Suppose a staff member asks:
"What tone should our social media captions use?"
The retrieval flow becomes:
User Question
│
▼
Knowledge Base
│
▼
Semantic Retrieval
│
├── Brand Guidelines chunk
├── Caption Bank chunk
└── Other relevant chunk
│
▼
Retrieved Context
Amazon Bedrock provides the Retrieve operation for retrieving source chunks relevant to a query.
This was important to my architecture because I wanted retrieval to remain an explicit stage of the application flow rather than hiding it behind a single black-box operation.
The number of chunks retrieved is another configuration decision.
For managed Knowledge Bases, numberOfResults controls the maximum number of source chunks returned. The actual number can be lower if fewer relevant results are available.
{
"retrievalConfiguration": {
"managedSearchConfiguration": {
"numberOfResults": 5
}
}
}
The important word is maximum.
Setting:
numberOfResults = 5
does not mean:
Always return exactly five chunks.
It means:
Return up to five relevant chunks.
More retrieved content is not automatically better.
If a question can be answered using two highly relevant chunks, adding several loosely related chunks may introduce unnecessary context into the model prompt.
So retrieval count becomes a tuning parameter influenced by:
Document structure
Chunking strategy
Question complexity
Retrieval quality
Model context requirements
Another important architectural distinction is separating retrieval from generation.
The Knowledge Base can retrieve the relevant source material, while the application remains responsible for passing that context to the foundation model.
For my architecture:
User Question
│
▼
Retrieve
│
▼
Relevant Chunks
│
▼
Application Orchestrator
│
▼
Claude Sonnet 4.5
│
▼
Generated Response
This separation gives the application control over how retrieved information is incorporated into the model request.
It also makes the system easier to debug.
Instead of treating the final answer as one opaque operation, I can inspect two separate stages:
Retrieval
↓
Did I retrieve the right information?
Generation
↓
Did the model produce the right answer from that information?
Those are fundamentally different failure modes.
The application was not intended to be an autonomous agent.
It was designed as an internal creative assistant with human review.
So I wanted the architecture to remain explicit:
Staff
│
▼
Application
│
▼
Retrieve Company Knowledge
│
▼
Construct Context
│
▼
Claude
│
▼
Draft Response
│
▼
Human Review
This creates a clear boundary between:
Retrieval
and:
Generation
That boundary is useful both architecturally and operationally.
If an answer is incorrect, I can investigate whether:
That is much more actionable than simply knowing that "the AI got it wrong."
Retrieval should also remain observable.
A useful RAG system should allow the application to understand where retrieved information came from.
The Retrieve response includes the retrieved source content and source-location information.
Generated Response
│
├── Source: Brand Guidelines
├── Source: Caption Bank
└── Source: Script Bank
For an internal assistant, this provides an additional layer of transparency.
The employee should not have to blindly trust a generated response. The application should be able to associate the response with the company information used to produce it.
The biggest advantage was a clearer responsibility boundary.
My application owns:
Authentication
API
Application Logic
Prompt Construction
Model Invocation
User Interface
Human Review
Access Control
Observability
The managed Knowledge Base handles much of:
Data Source Integration
Document Ingestion
Parsing
Chunking
Embedding Generation
Vector Indexing
Retrieval
That means fewer retrieval-specific components become part of my application code and operational surface.
Choosing a managed abstraction also means giving up some low-level control.
That tradeoff is important.
| Lower-level / Self-managed | Managed Knowledge Base |
|---|---|
| More direct control over vector infrastructure | Higher-level AWS abstraction |
| More control over retrieval implementation | Bedrock-managed retrieval |
| More responsibility for ingestion | Managed ingestion workflow |
| More infrastructure to configure | Less application-owned infrastructure |
| More flexibility at the infrastructure layer | Less low-level control |
| More operational responsibility | Lower operational overhead |
So I would not describe the managed Knowledge Base as universally better.
The appropriate architecture depends on the application's requirements.
If an application required highly customized vector infrastructure or retrieval behavior, a lower-level architecture could make sense.
For this project, I valued the managed ingestion and retrieval pipeline more than having complete ownership of the underlying vector infrastructure.
After working through those tradeoffs, the AI architecture became:
Company Documents
│
▼
Amazon S3
│
▼
S3 Data Source
│
▼
Ingestion / Sync
│
┌─────────────┴─────────────┐
│ │
Parsing Chunking
│ │
└─────────────┬─────────────┘
│
▼
Embeddings
│
▼
Vector Index
│
▼
Retrieval
│
▼
Relevant Context
│
▼
Application Orchestrator
│
▼
Claude Sonnet 4.5
│
▼
Draft Response
│
▼
Human Review
The broader application architecture was:
Staff User
│
▼
Cognito
│
▼
API Gateway
│
▼
Chat Orchestrator
│
┌──────────┴──────────┐
│ │
▼ ▼
Bedrock Knowledge Claude Sonnet 4.5
Base │
│ │
└──── Retrieved ───────┘
Context
│
▼
Draft Response
│
▼
Human Review
And the knowledge pipeline was:
Company Documents
│
▼
S3
│
▼
Knowledge Base Data Source
│
▼
Ingestion
│
├── Parsing
├── Chunking
├── Embeddings
└── Indexing
│
▼
Retrieval
If I had to summarize the decision:
I chose Amazon Bedrock Managed Knowledge Bases because I wanted AWS to handle much of the operational complexity of the RAG ingestion and retrieval pipeline while my application focused on authentication, orchestration, generation, and the user experience.
That is a more useful way to think about managed services than simply saying:
"AWS has a service that does this for me."
The important architectural question is:
Where should responsibility live?
The biggest lesson was that cloud architecture is not simply about choosing the most powerful service or building everything yourself.
It is about defining boundaries.
S3
↓
Vector Database
↓
LLM
After building the system, my mental model became:
SOURCE
│
▼
S3
│
▼
INGESTION
│
┌────────┼────────┐
▼ ▼ ▼
Parsing Chunking Embedding
│
▼
Index
│
▼
Retrieval
│
▼
Application
Context
│
▼
Claude
│
▼
Response
│
▼
Human Review
That shift was one of the most valuable parts of the architecture decision.
I stopped thinking about RAG as simply:
"Put documents in a vector database and ask an LLM questions."
Instead, I started thinking about it as a pipeline with explicit responsibilities:
Storage
↓
Ingestion
↓
Representation
↓
Retrieval
↓
Context
↓
Generation
↓
Review
That mental model made it much easier to reason about where managed AWS infrastructure could provide value and where application-level control was still necessary.
Amazon Bedrock Managed Knowledge Bases was not simply a shortcut for me.
It was an architectural choice.
I could have owned more of the ingestion and vector-search infrastructure myself. Instead, I chose to delegate that responsibility to a managed AWS service and keep my application focused on the parts that were specific to the product.
The resulting responsibility boundaries were:
Amazon S3
→ Source documents
Bedrock Knowledge Base
→ Ingestion + retrieval
Application
→ Orchestration + context construction
Claude
→ Generation
Human
→ Final review
For this project, that division of responsibility gave me the balance I was looking for: enough control over application behavior without unnecessarily taking ownership of infrastructure that AWS could manage for me.