# Building an Internal AI Assistant on AWS: A Production-Ready RAG Architecture with Amazon Bedrock > Source: > Published: 2026-09-21 12:02:09+00:00 # Building an Internal AI Assistant on AWS: A Production-Ready RAG Architecture with Amazon Bedrock AI is becoming an increasingly important part of how organizations handle day to day work. But introducing AI into an organization is not just about connecting an application to an LLM and allowing people to start asking questions. For an internal AI assistant to be useful, it needs access to the or AI is becoming an increasingly important part of how organizations handle day to day work. But introducing AI into an organization is not just about connecting an application to an LLM and allowing people to start asking questions. For an internal AI assistant to be useful, it needs access to the organization's own information. It also needs authentication, access control, protection around AI input and output, logging, and a clear separation between the AI system and the people responsible for reviewing its responses. In this project, I built an internal AI assistant on AWS that allows authenticated staff to interact with an AI assistant using the organization's internal documents as its knowledge source. The system uses: AWS Amplify for hosting the frontend Amazon Cognito for staff authentication Amazon API Gateway for the API layer AWS Lambda for application logic Amazon Bedrock Knowledge Bases for retrieval augmented generation Amazon Bedrock for the foundation model Amazon Bedrock Guardrails for controlling model input and output Amazon S3 for storing internal documents Amazon CloudWatch for logging and monitoring AWS IAM for permissions and access control Terraform for infrastructure provisioning The goal was not simply to get an AI model to generate a response. I wanted to build the complete application around the model. A staff member logs in, sends a request through the application, API Gateway validates the request, Lambda coordinates the AI workflow, relevant information is retrieved from the organization's documents through a managed Bedrock Knowledge Base, the model generates a response using that context, and the response is returned to the authenticated staff member. The system was also designed around a human review process. The assistant generates drafts and responses for staff to review. It does not automatically publish content or take actions on behalf of the organization. Building this project gave me practical experience working with multiple AWS services as one system rather than learning each service in isolation. Building and hosting a static frontend on AWS with AWS Amplify Implementing authentication with Amazon Cognito Protecting an API with JWT authentication Building a serverless backend with Lambda Connecting Lambda to Amazon Bedrock Building a RAG workflow using Bedrock Knowledge Bases Using S3 as a document source Applying Bedrock Guardrails to AI requests and responses Debugging IAM permission issues Working with Bedrock model access and inference profiles Monitoring Lambda execution with CloudWatch Managing the infrastructure with Terraform Designing an AI workflow where humans remain responsible for the final output There are many areas where employees repeatedly need to search through documents, look at previous work, check company guidelines, draft content, or find information that already exists somewhere inside the organization. The problem is that having the information does not necessarily mean that it is easy to use. Important documents can be spread across different files and folders. Previous scripts, captions, proposals, brand guidelines and other internal resources can become difficult to search through manually. This was the problem I wanted to address with this project. The idea was to build an internal AI assistant that could give staff a simpler way to interact with the organization's existing knowledge. Instead of manually searching through multiple documents before starting a task, a staff member could ask the assistant a question and have the system retrieve relevant information from the organization's internal knowledge base. That led me to use a Retrieval Augmented Generation architecture with Amazon Bedrock Knowledge Bases. At a high level, internal documents are stored in Amazon S3 and connected to a Bedrock Knowledge Base. When a staff member submits a request, the application retrieves relevant information from the knowledge base and uses that information as context for generating the response. But building the complete system required more than the RAG component. I also needed to solve authentication, API security, permissions, input and output controls, application logic, logging and infrastructure provisioning. That is what made this project interesting from an engineering perspective. The assistant was designed for internal staff who need to work with the organization's existing content and information. The knowledge base included areas such as Brand guidelines Previous scripts Captions Client proposals The documents were organized in Amazon S3 using separate prefixes: brand-guidelines/ past-scripts/ captions/ proposals/ For example, a staff member could ask the assistant to create a draft based on the organization's existing style and previous work. Instead of relying only on the model's general knowledge, the application could retrieve relevant internal content and use it as context. This gives the system two important components. The model provides the language generation capability. The organization's documents provide the context. The application brings the two together. Before building the system, I had to answer a few practical questions. The assistant was intended for internal use, so I needed an authentication mechanism that would prevent unauthenticated users from accessing the application API. I used Amazon Cognito User Pools for this. The browser needed to communicate with an API, but I did not want the frontend directly interacting with Bedrock or having broad AWS permissions. I used Amazon API Gateway as the API layer and configured a JWT authorizer using tokens issued by Cognito. A foundation model would not automatically know the organization's private documents. I needed a retrieval layer that could search the organization's content and provide relevant information to the model. This is where Amazon Bedrock Knowledge Bases came into the architecture. Amazon S3 became the document repository and the source for the Knowledge Base. An internal AI assistant still needs controls around what users can submit and what the system can return. I used Amazon Bedrock Guardrails for this. With several managed services involved, I needed visibility into what was happening during each request. Amazon CloudWatch provided the logging and monitoring needed to troubleshoot the backend. I wanted the infrastructure to be reproducible rather than relying entirely on manual configuration through the AWS console. I used Terraform to provision and manage the AWS resources. The final architecture connects the staff member's browser to several AWS managed services. The main request path is: Staff → Amplify → Cognito → API Gateway → Lambda → Guardrails → Knowledge Base → Bedrock Model → Guardrails → Response The supporting data path is: S3 → Bedrock Knowledge Base → Document processing and indexing CloudWatch provides observability across the backend, while IAM controls access between the AWS services. Here's the complete architecture. The staff member opens the frontend hosted on AWS Amplify. The staff member authenticates through Amazon Cognito. Cognito issues the authentication tokens required by the application. The frontend sends the authenticated request to Amazon API Gateway. API Gateway validates the JWT before allowing the request to reach Lambda. AWS Lambda receives the request and acts as the orchestration layer. The request passes through the configured Bedrock Guardrails. Lambda retrieves relevant information from the Bedrock Knowledge Base. The retrieved information is provided as context for the model generation step. Amazon Bedrock generates the response. The generated response passes through the output controls. Lambda formats the response and returns it through API Gateway. The frontend displays the response to the authenticated staff member. The important part of this architecture is the separation of responsibilities. The browser handles the user interface, Cognito handles authentication, API Gateway protects the API boundary, Lambda coordinates the workflow, Bedrock handles retrieval and generation, S3 stores the source documents, and CloudWatch provides visibility into the system. I started with a simple frontend. I did not need a large frontend framework for this application because the main purpose of the interface was to provide staff with a simple way to authenticate, enter a prompt and view the generated response. The frontend was built using: HTML CSS JavaScript The application contained the basic interface required for the assistant. The important part here was that the frontend was intentionally kept simple. Most of the complexity of the application lives in the backend and AWS services. The browser is responsible for providing the user interface and communicating with the API. It should not be responsible for directly accessing the organization's S3 documents, invoking Bedrock models or managing broad AWS permissions. I hosted the static frontend using AWS Amplify. The deployment model was straightforward. The frontend files were deployed to Amplify and served to staff through the application's web interface. This also gave me a clean separation between the presentation layer and the backend. The browser communicates with the API. The API communicates with the backend services. The backend communicates with Bedrock and the organization's internal knowledge. Screenshot: Frontend of the internal ai assistant deployed with amplify The next problem was authentication. Because this was an internal application, I needed to make sure that only authenticated staff could access the assistant. I used Amazon Cognito User Pools for this. The Cognito User Pool became the identity layer for the application. I configured the application so that users could authenticate through Cognito's managed login experience. The basic authentication flow was: Staff member ↓ Amazon Cognito Managed Login ↓ Successful authentication ↓ Authentication tokens ↓ Frontend The frontend then uses the authentication information when communicating with the backend API. I wanted authentication to happen before the request reached the application logic. This meant the Lambda function did not need to implement the basic responsibility of determining whether a request was authenticated. Instead, API Gateway could validate the JWT issued by Cognito. The request path therefore became: Browser ↓ Cognito authentication ↓ JWT ↓ API Gateway ↓ JWT validation ↓ Lambda This gave me a cleaner separation of responsibilities. Cognito handled identity. API Gateway handled API-level authentication. Lambda handled application logic. Screenshot: Cognito User Pool overview Once authentication was in place, I needed an API for the frontend to communicate with the backend. I used Amazon API Gateway with an HTTP API. The application exposed a chat endpoint for the assistant. The important part of this configuration was the JWT authorizer. The API Gateway route was configured to validate tokens issued by the Cognito User Pool. That meant a request without a valid token should not reach the Lambda function. I tested this during implementation. When I attempted to access the API without valid authentication, the API returned: 401 Unauthorized This was an important test because it confirmed that the JWT authorizer was enforcing authentication at the API layer. Screenshot: API Gateway route and JWT authorizer After API Gateway validates the request, the request is passed to AWS Lambda. I used a Python Lambda function as the chat orchestrator. The Lambda function became the central piece connecting the application to the Bedrock services. Instead of allowing the frontend to directly communicate with Bedrock, the Lambda function controlled the workflow. Conceptually, the Lambda function is responsible for: Receive request ↓ Validate/process input ↓ Apply input guardrail ↓ Retrieve relevant knowledge ↓ Generate response ↓ Apply output guardrail ↓ Return response This was one of the most important architectural decisions in the project. The frontend does not need to know how the Bedrock Knowledge Base works. It does not need to know the model configuration. It does not need direct permissions to invoke the model. It simply sends a request to the API. Lambda handles the orchestration. Screenshot: Lambda configuration The assistant needed access to the organization's existing information. I used Amazon S3 as the document repository. I organized the documents into separate prefixes: brand-guidelines/ past-scripts/ captions/ proposals/ This made the document source easier to organize and gave the knowledge base a structured location from which to retrieve information. The important point here is that S3 is not the component generating the answer. S3 stores the source documents. The Bedrock Knowledge Base uses those documents as its data source. That distinction becomes important when looking at the RAG workflow. Screenshot: S3 bucket and prefixes This was the part of the architecture that allowed the assistant to work with the organization's internal knowledge. I created a managed Amazon Bedrock Knowledge Base and connected it to the S3 document source. The documents then go through the knowledge base ingestion process. At a high level: S3 documents ↓ Knowledge Base data source ↓ Ingestion ↓ Document processing ↓ Chunking ↓ Embeddings ↓ Vector representation ↓ Retrieval When a staff member asks a question, the system can retrieve relevant chunks from the indexed content. The model can then use those retrieved chunks when generating its response. The first ingestion test did not work as expected. The issue was not with the PDF itself. The problem was the relationship between the S3 object location and the configured data source. The knowledge base was configured to look at a particular S3 prefix, while the test document was not located where the data source expected it. After correcting the S3 location and running ingestion again, the ingestion job completed successfully. The result showed: Status: COMPLETE Scanned: 1 Indexed: 1 Failed: 0 That was a useful debugging lesson because it showed that a RAG system can fail before the model is ever involved. If the knowledge base does not ingest the documents correctly, there is nothing useful for the model to retrieve. Screenshot: Knowledge Base configuration The next part of the implementation was controlling the AI interaction. I used Amazon Bedrock Guardrails to add controls around the assistant's input and output. The Lambda orchestration flow therefore became: User prompt ↓ Input Guardrail ↓ Knowledge retrieval ↓ Model generation ↓ Output Guardrail ↓ Final response The guardrail configuration was an important part of the project because it demonstrated that the AI model was not being treated as an unrestricted component. I also ran into an issue with Guardrail versioning. I had changed the guardrail configuration and expected the active version to reflect the new configuration. However, the existing guardrail version was immutable. A configuration change did not automatically mutate the already-created version. During testing, an address was still being detected even after I had removed that configuration from the desired Terraform definition. The test returned: GUARDRAIL_INTERVENED and identified the address category. This made it clear that the Lambda was still referencing the older guardrail version. I resolved this by replacing the guardrail version through Terraform and then verifying that the Lambda was using the new version. The Terraform operation was: terraform apply -replace=aws_bedrock_guardrail_version.creative_assistant This was one of the more useful lessons from the project. With versioned AWS resources, changing the Terraform configuration does not always mean that an existing immutable version will change in place. You need to understand which resource is actually being referenced at runtime. Screenshot: Guardrail overview After the knowledge retrieval layer was working, I connected the Lambda orchestrator to the Bedrock foundation model. The Lambda function needed permissions to invoke the model. This introduced another issue during testing. The application initially reached the knowledge retrieval stage successfully, but model generation failed because the Lambda execution role did not have the required bedrock:InvokeModel permission for the model invocation path. This was visible in the CloudWatch logs. The useful part of the debugging process was that the logs showed that the earlier parts of the workflow were already working. The input guardrail had passed. The knowledge base had returned a result. The failure happened when the application attempted to invoke the model. That narrowed the problem down to the model invocation and IAM configuration rather than the entire application. I updated the IAM permissions and continued testing. I also encountered an issue with model throughput. The base Claude Sonnet 4.5 foundation model identifier did not support the on-demand invocation pattern I was using. The solution was to use the appropriate US inference profile for Claude Sonnet 4.5. The model configuration was then updated in Terraform and the Lambda environment. This was another useful AWS lesson. Having access to a model in Amazon Bedrock does not automatically mean that every model identifier can be invoked using every invocation method. The model ID, inference profile and IAM permissions all need to line up with the way the application is invoking the model. IAM was used throughout the architecture to control which AWS resources each component could access. The frontend does not need permission to invoke Bedrock. The frontend does not need permission to read the internal S3 documents. The Lambda function is the component that needs permissions to interact with the services required for the AI workflow. This is where the principle of least privilege becomes important. Instead of giving the application broad AWS permissions, I created permissions around the actual operations required by the system. The Lambda execution role needed access to the relevant Bedrock operations and supporting resources. The Knowledge Base also needed the permissions required to access its configured data source. This separation helped keep the trust boundaries clear. Frontend ↓ API Gateway ↓ Lambda ↓ Bedrock services The browser never receives broad AWS permissions that would allow it to directly interact with the internal infrastructure. Once several AWS services are connected together, debugging becomes much easier when there is a clear place to inspect what happened. I used Amazon CloudWatch for Lambda logging and monitoring. CloudWatch became particularly useful during the problems I encountered while testing the system. For example, one request initially returned: 500 Unable to generate the draft. From the frontend alone, that error did not tell me much. The CloudWatch logs provided the actual execution path. I could see that: Input Guardrail ↓ Passed Knowledge Base ↓ Retrieved result Model invocation ↓ Failed That completely changed the debugging process. Instead of guessing whether Cognito, API Gateway, Lambda, the Knowledge Base or the model was responsible, I could follow the execution path and identify the stage where the request failed. Screenshot: CloudWatch logs I used Terraform to provision and manage the AWS infrastructure for the project. By this point, the application involved several services, including Cognito, API Gateway, Lambda, S3, Bedrock Knowledge Bases, Bedrock Guardrails, IAM and CloudWatch. Creating and configuring all of these manually through the AWS console would make the environment harder to reproduce and maintain. I therefore defined the infrastructure as code. The Terraform configuration was split into separate files based on the resources they managed, including: terraform/ ├── api-gateway.tf ├── cloudwatch.tf ├── cognito.tf ├── guardrail.tf ├── iam.tf ├── knowledge-base.tf ├── lambda.tf ├── s3.tf ├── variables.tf └── outputs.tf This also became useful during debugging. For example, when I discovered that the Claude Sonnet 4.5 model invocation required a different inference profile and corresponding IAM configuration, I could make the changes in Terraform and use: terraform plan to see exactly what Terraform intended to change before applying it. I also ran into an issue with the Bedrock Guardrail version. The configuration had changed, but the existing guardrail version was immutable, so changing the Terraform configuration alone did not update the version being used by Lambda. I resolved this by explicitly replacing the guardrail version: terraform apply -replace=aws_bedrock_guardrail_version.creative_assistant This was one of the practical benefits of using Terraform in the project. Infrastructure changes, including changes required while debugging, could be tracked and reproduced rather than relying entirely on manual console changes. Terraform was therefore not part of the runtime request path. It was the infrastructure layer used to provision and manage the AWS resources that made the application possible. Once the individual components were configured, I tested the application as a complete system. The final workflow was: Staff member ↓ AWS Amplify frontend ↓ Amazon Cognito authentication ↓ JWT ↓ Amazon API Gateway ↓ JWT validation ↓ AWS Lambda ↓ Amazon Bedrock Guardrail ↓ Amazon Bedrock Knowledge Base ↓ Relevant internal context ↓ Claude Sonnet 4.5 ↓ Output Guardrail ↓ Lambda response ↓ API Gateway ↓ Frontend ↓ Staff member CloudWatch provided visibility into the backend execution while Terraform provided the infrastructure management layer around the application. The important thing about the architecture is that each service has a specific responsibility. The frontend provides the interface. Cognito handles identity. API Gateway protects the API boundary. Lambda orchestrates the workflow. S3 stores the source documents. Bedrock Knowledge Bases handles retrieval. Bedrock provides the model. Guardrails provide AI safety controls. IAM controls permissions. CloudWatch provides observability. Terraform manages the infrastructure. This gave me a reproducible way to manage the environment instead of relying entirely on changes made manually through the AWS console. The biggest lesson from this project was that building an AI application is not simply a matter of choosing a model. The model was only one component of the system. Most of the engineering work was around everything surrounding it. Authentication had to work. The API had to reject unauthenticated requests. The documents had to be ingested correctly. The knowledge base had to return useful context. The Lambda execution role needed the correct permissions. The model invocation method had to match the selected model and inference profile. Guardrail versions needed to be managed correctly. And when something failed, CloudWatch needed to provide enough information to understand where the failure occurred. That was what made the project useful to me from a cloud engineering perspective. I was able to work with several AWS services as one system instead of treating each service as an isolated topic. The final result was an internal AI assistant where authenticated staff can interact with an AI system grounded in the organization's own content, while keeping the actual content generation process under human review. This project started with a relatively simple idea. Give staff an easier way to work with information that already exists inside the organization. The implementation ended up involving authentication, API security, serverless computing, object storage, managed RAG, foundation models, guardrails, IAM, monitoring and infrastructure as code. That was the part I found most valuable. The AI model is only one piece of the application. The real engineering challenge is building the system around it in a way that is secure, observable, reproducible and useful to the people who are going to use it. ## Key Takeaways - •AI is becoming an increasingly important part of how organizations handle day to day work - •This story was reported by **Dev.to** , covering developments in the**dev** space. - •AI advancements continue to reshape industries — read the full article on Dev.to for complete coverage. 📖 Continue reading the full article: [Read Full Article on Dev.to →](https://dev.to/duubemmm/building-an-internal-ai-assistant-on-aws-with-amazon-bedrock-and-managed-rag-26bp)