Implement vector-prompt document classification using Amazon Bedrock

Amazon Web Services (AWS) announced a new multi-agent solution using the Strands Agents SDK on Amazon Bedrock to classify insurance documents, combining Anthropic's Claude Haiku 4.5 with Amazon Titan Multimodal Embeddings. The system orchestrates three agents—Document Analysis, Vector Similarity Search, and Validation—to improve accuracy over single-model approaches, addressing challenges like similar-looking documents with different purposes.

Artificial Intelligence /blogs/machine-learning/ Implement vector-prompt document classification using Amazon Bedrock Vector-prompt classification on Amazon Bedrock /bedrock/ helps insurance companies accurately classify thousands of daily documents: policies, affidavits, endorsements, and regulatory forms, for compliance, claims, and customer service. Manual classification is time-consuming and error-prone, while traditional automated approaches struggle with documents that look similar but serve different purposes. A policy endorsement and a regulatory affidavit might contain similar terminology, yet misclassifying them can lead to compliance violations or processing delays. This post demonstrates how you can build a multi-agent solution using the Strands Agents SDK https://strandsagents.com/latest/ . The solution orchestrates three specialized agents: a Document Analysis Agent for textual reasoning, a Vector Similarity Search Agent for layout pattern recognition, and a Validation Agent for quality assurance. Each agent operates autonomously within its expertise, then collaborates through an Orchestrator to deliver results. You will learn how to implement this multi-agent architecture for your own document classification needs, with code examples and technical guidance. This multi-agent approach combines the advanced reasoning capabilities of Anthropic’s Claude Haiku 4.5 /bedrock/anthropic/ with the visual pattern recognition of Amazon Titan Multimodal Embeddings https://docs.aws.amazon.com/bedrock/latest/userguide/titan-models.html available on Amazon Bedrock /bedrock to achieve better classification accuracy. Solution overview The solution architecture combines multiple specialized AI agents, each optimized for specific aspects of document analysis, working together through coordinated orchestration. This multi-agent approach addresses the limitations of single-model classification by using the unique strengths of different foundation models and techniques available through Amazon Bedrock. The following diagram illustrates the multi-agent approach: Multi-agent coordination with the Strands Agents SDK In our testing, single-model approaches struggled with edge cases and complex documents that require both textual and visual analysis. Multi-agent systems address this by breaking down the classification task into specialized subtasks, with each agent focusing on its area of expertise. We chose the Strands Agents SDK https://strandsagents.com/ because it implements the agents as tools https://strandsagents.com/docs/user-guide/concepts/multi-agent/agents-as-tools/ and our classification system needs an orchestrator that can invoke specialized agents as callable tools. The Validation Agent calls each specialist, compares their classifications, and resolves disagreements without custom orchestration code. This pattern offers several advantages: Modularity : Each agent can be developed, tested, and improved independently. Transparency : Every agent provides reasoning for its decisions, creating an audit trail. Flexibility : New agents can be added without restructuring the entire system. Reliability : The orchestrator handles agent coordination, error handling, and result synthesis. Architecture components At the core of the architecture is Validation Agent , which acts as an orchestrator and implements the agents as tools pattern using the Strands Agents SDK. This agent provides quality assurance through cross-validation and confidence scoring. It compares the outputs from both the Document Analysis Agent and Vector Similarity Search Agent . It identifies areas of agreement and disagreement, then generates a final classification with an associated confidence score. This validation step helps the system maintain high accuracy while flagging edge cases for human review. The Validation Agent coordinates with two specialized agents: Document Analysis Agent : This agent uses Anthropic’s Claude Haiku 4.5 /bedrock/anthropic/ on Amazon Bedrock /bedrock/ for advanced textual reasoning and legal language interpretation. Claude excels at understanding complex documents, extracting key information, and identifying subtle patterns in text that indicate document type. The agent analyzes document content, metadata, and linguistic features to generate classification hypotheses. Vector Similarity Search Agent : This agent uses Amazon Titan Multimodal Embeddings G1 https://docs.aws.amazon.com/bedrock/latest/userguide/titan-models.html to convert documents into high-dimensional vector representations for visual similarity search. Claude Haiku 4.5 excels at understanding document content: analyzing text, extracting key information, and identifying linguistic patterns. The Vector Similarity Search Agent complements this by focusing on visual and structural characteristics. This agent captures how documents look rather than what they say. It identifies formatting patterns like form layouts, table structures, and formatting conventions that distinguish document types even when textual content varies. The agent uses FAISS https://faiss.ai/index.html Facebook AI Similarity Search for efficient vector similarity search, which supports rapid comparison against known document templates. By combining textual and similarity analysis with built-in validation, this multi-agent architecture achieves higher classification accuracy and provides reliable confidence scores for automated decision-making. In the next sections, you will learn how to implement each component and deploy the complete solution. Prerequisites To follow along with this walkthrough, you will need the following: AWS account and permissions - An active AWS account with permissions to access Amazon Bedrock. - AWS Identity and Access Management IAM permissions to create and invoke foundation models FMs . - Access to Anthropic’s Claude Haiku 4.5 https://docs.aws.amazon.com/bedrock/latest/userguide/model-card-anthropic-claude-haiku-4-5.html and Amazon Titan Multimodal Embeddings https://docs.aws.amazon.com/bedrock/latest/userguide/model-card-amazon-titan-multimodal-embeddings-g1.html models. For model availability by AWS Region, see Supported models by AWS Region in Amazon Bedrock https://docs.aws.amazon.com/bedrock/latest/userguide/models-regions.html . Development environment - Python 3.14 or later installed. - AWS CLI version 2.0 or later, configured with your credentials. - An integrated development environment IDE or text editor such as VS Code or PyCharm . - Git for cloning the sample repository. Software and libraries - Strands Agents SDK installation instructions provided in the walkthrough . - FAISS library for vector similarity search. Note: This walkthrough uses AWS services that might incur costs. Make sure to review the pricing for Amazon Bedrock /bedrock/pricing/ and follow the cleanup instructions at the end to avoid ongoing charges. Implementing the multi-agent document classification system Let’s walk through implementing each component of the multi-agent system. The complete code is available in our GitHub repository. Step 1: Configure the foundation models First, configure access to the Claude model through an Amazon Bedrock inference profile. This provides consistent performance and availability across Regions. Tip: For lower latency and higher availability HA , you can use a cross-Region inference profile by adding a geographic prefix us., eu., or ap. that matches your deployment Region to the model ID. Step 2: Create the Document Analysis Agent The Document Analysis Agent specializes in textual content analysis using the advanced reasoning capabilities of Claude Haiku 4.5. The agent uses structured output to return consistent, machine-parseable classification results. Step 3: Build the Vector Similarity Search Agent The Vector Similarity Search Agent uses Amazon Titan Multimodal Embeddings to analyze document layout and visual characteristics. This agent performs FAISS vector similarity search using perform vector classification to match documents against pre-trained visual patterns stored in the vector database. Step 4: Implement the Validation Agent The Validation Agent coordinates the specialist agents using the agents as tools pattern. The Validation Agent cross-checks results from both specialist agents for quality and consistency. Step 5: Classify documents With the components in place, here’s how the MultiAgentDocumentClassifier processes a document end-to-end: To use the classifier: The system processes each document through specialist agents, synthesizing their analyses into a final classification with comprehensive reasoning. Understanding the results Model selection For this workload, we selected Claude Haiku 4.5 on Amazon Bedrock as our inference model. Haiku 4.5 met our accuracy requirements while delivering lower latency and cost, averaging 19.3 seconds per document classification at 93 percent confidence. This makes it well-suited for production workloads where speed and cost-efficiency are priorities without sacrificing correctness. Common approach comparison We benchmarked the multi-agent system against three commonly used AWS approaches. The goal was straightforward: identify the most straightforward approach that meets accuracy requirements for compliance-sensitive workloads. Amazon Textract and Amazon Comprehend are purpose-built for text extraction and entity recognition. They excel at those tasks. However, they weren’t designed for multi-class document classification where documents share overlapping legal terminology. Both achieved only 25 percent accuracy on our test set. Amazon Bedrock Data Automation BDA performed significantly better at 70 percent, correctly handling most affidavits and miscellaneous documents, but still misclassified nearly one in three documents overall. The multi-agent system was the only approach to achieve 100 percent accuracy across all document classes. Approach | Accuracy | Avg Time | Complexity | Cost | | | 1 | Amazon Textract + Keywords | 25% | 2.88s | Low | Low | | 2 | Amazon Comprehend + Entities | 25% | 3.26s | Medium | Low | | 3 | Amazon Bedrock Data Automation | 70% | 25.7s | Medium | Medium | | 4 | Multi-Agent System | 100% | 23.3s | High | Medium | Accuracy by class and approach The accuracy gap is most pronounced on policy documents: dense legal text with terminology that overlaps heavily with other document classes. Only the multi-agent system classified these correctly. Affidavits showed a similar pattern: BDA handled them well, but Amazon Textract and Amazon Comprehend couldn’t distinguish them from adjacent legal document types. Miscellaneous documents, which contain distinct structural markers, were classified correctly by all approaches. Class | Count | Amazon Textract | Amazon Comprehend | BDA | Multi-Agent | | | 1 | Policy | 8 | 25% | 25% | 25% | 100% | | 2 | Affidavit | 10 | 10% | 10% | 100% | 100% | | 3 | Miscellaneous | 2 | 100% | 100% | 100% | 100% | In our testing with a limited evaluation scope of 20 documents across 3 classes, the multi-agent system achieved 100 percent accuracy, representing a 30 percent improvement over the next best approach BDA at 70 percent . Production accuracy might vary with larger and more diverse document sets. Amazon Textract and Amazon Comprehend are purpose-built for text extraction and entity recognition respectively, and excel at those tasks. However, they weren’t designed for nuanced multi-class document classification where documents share overlapping terminology. These results reflect an initial run without fine-tuning or custom classification logic. With additional configuration, accuracy could improve for specific use cases. The multi-agent system trades processing time about 23 seconds per document for significantly higher accuracy, making it well-suited for use cases where classification errors carry compliance or financial risk. Within the multi-agent framework, Claude Haiku 4.5 matched the accuracy of Claude Sonnet 4.5 while running 17 percent faster. This makes it the better choice for production workloads at lower latency and cost. Production safeguards Amazon Bedrock Guardrails /bedrock/guardrails/ provides content filtering and safety controls for model interactions. Configurable policies support denied topics, content filters, word filters, and sensitive information redaction for responsible AI deployment at the application layer. For a document classification pipeline processing sensitive insurance documents, explicit controls are required to make sure outputs remain accurate, appropriate, and auditable. Without these controls, a classification pipeline can produce hallucinated categories, leak sensitive data in logs or downstream systems, or make confident but incorrect decisions that violate compliance requirements. When you deploy this multi-agent classification system, we recommend the following best practices: - Apply personally identifiable information PII redaction filters to help prevent policyholder information from propagating into classification logs. - Configure topic-denial policies to constrain agents to their classification scope. - Turn on Amazon Bedrock model invocation logging to capture the full request-response chain, including guardrail intervention events, for auditing. In our solution, the requires human review flag from the Validation Agent triggers automatic escalation when confidence falls below a defined threshold. This makes sure uncertain classifications reach a human reviewer rather than propagating downstream. Cleaning up To avoid incurring future charges, delete the resources created during this walkthrough: Delete local resources: - Remove the FAISS vector database files hybrid docs.vdb from your local environment. - Delete the test PDF documents you uploaded for classification. - Clear the Python virtual environment if you created one specifically for this project. Delete AWS resources: - If you stored training documents in Amazon Simple Storage Service Amazon S3 , delete the S3 bucket and its contents. - Clear the Amazon CloudWatch logs generated during testing. - Review your Amazon Bedrock usage in the AWS Management Console to confirm there are no ongoing invocations. Note : Amazon Bedrock charges are based on model invocations, so there are no persistent resources to delete. However, reviewing your usage helps you understand the costs incurred during testing. For detailed cleanup instructions and scripts, see the cleanup section in our GitHub repository https://github.com/aws-samples/sample-vector-prompt-classification/ . Conclusion This post demonstrated how you can improve document classification accuracy through multi-agent workflows using Amazon Bedrock foundation models and the Strands Agents SDK. By orchestrating three specialized AI agents, you can achieve more accurate classification results that combine the strengths of both textual and visual analysis. The solution uses a Document Analysis Agent powered by Anthropic’s Claude Haiku 4.5, a Vector Similarity Search Agent using Amazon Titan Multimodal Embeddings, and a Validation Agent for quality assurance. This multi-agent approach offers several advantages over traditional single-model classification systems. The specialized agents work autonomously within their domains of expertise, then collaborate through the Orchestrator to deliver results that are both accurate and explainable. You can begin by implementing the Document Analysis Agent with your own document types, then add the Vector Similarity Search Agent to improve accuracy on visually distinctive documents. Experiment with different confidence thresholds for human review to balance automation with quality assurance. Additional Resources: - Explore the complete implementation in our GitHub repository https://github.com/aws-samples/sample-vector-prompt-classification/ . - Learn more about foundational models available on Amazon Bedrock /bedrock/model-choice/ . - Read the Strands Agents SDK https://strandsagents.com/latest/ documentation for advanced orchestration patterns. - Check out related posts on intelligent document processing /blogs/machine-learning/accelerate-intelligent-document-processing-with-generative-ai-on-aws/ and multi-agent architectures /blogs/machine-learning/using-strands-agents-to-create-a-multi-agent-solution-with-metas-llama-4-and-amazon-bedrock/ .