{"slug": "building-agentic-workflows-with-sagemaker-ai-and-bedrock-agentcore", "title": "Building agentic workflows with SageMaker AI and Bedrock AgentCore", "summary": "Amazon Web Services (AWS) published a guide for building agentic workflows that combine OpenAI-compatible endpoints on Amazon SageMaker AI with Amazon Bedrock AgentCore runtime, enabling specialized agents to use different models. The architecture deploys Qwen 3.5 9B on SageMaker AI, integrates it with Strands Agents multi-agent system alongside Claude Haiku 4.5 and Claude Sonnet 4.6 on Amazon Bedrock, and ships the workflow to Amazon Bedrock AgentCore runtime. The post details integration mechanics, including token-level observability from SageMaker endpoints, and provides source code in a GitHub repository.", "body_md": "[Artificial Intelligence](/blogs/machine-learning/)\n\n# Building agentic workflows with SageMaker AI and Bedrock AgentCore\n\nA common challenge in building agentic workflows is mixing managed foundation models (FMs) with your own cost-optimized or domain-specific models, without rewriting your agent framework to do it. In this post, we show you how to combine OpenAI-compatible endpoints on Amazon SageMaker AI with Amazon Bedrock AgentCore runtime, a capability of Amazon Bedrock AgentCore, and its managed deployment. Specialized agents can collaborate on complex tasks while each uses the model best suited to its job. This combination gives you cost optimization, data residency, and model flexibility in a single production-ready architecture.\n\nWe walk through deploying Qwen 3.5 9B on Amazon SageMaker AI, integrating it into a [Strands Agents](https://strandsagents.com/) multi-agent system alongside models on [Amazon Bedrock](/bedrock/), and shipping the entire workflow to Amazon Bedrock AgentCore runtime. The focus is on the integration mechanics including how to get token-level observability from SageMaker endpoints, which Strands doesn’t provide by default.\n\n## Solution overview\n\nThe architecture connects three model-hosting paths through a single Amazon Bedrock AgentCore container:\n\n**Orchestrator agent (Claude Haiku 4.5 on Bedrock)**– Classifies user intent and routes tasks through Global cross-Region inference.** Budget agent (Claude Sonnet 4.6 on Bedrock)**– Handles 50/30/20 budget breakdowns with structured Pydantic output.** Financial analysis agent (Qwen 3.5 9B on Amazon SageMaker AI)**– Stock analysis and portfolio construction using tool-calling.\n\nAmazon Bedrock model availability varies by AWS Region. See [Supported models by AWS Region in Amazon Bedrock](https://docs.aws.amazon.com/bedrock/latest/userguide/models-regions.html).\n\nA user request enters the orchestrator agent running inside the Amazon Bedrock AgentCore runtime. The orchestrator uses the *agents as tools* pattern from Strands Agents to route the request to either the budget agent or the financial analysis agent. Both specialized agents call their respective models. The budget agent invokes Claude Sonnet 4.6 through Amazon Bedrock, and the financial analysis agent invokes Qwen 3.5 9B through a SageMaker AI real-time endpoint using the OpenAI-compatible API. Results flow back through the orchestrator to the user. For the complete source code, see the [accompanying GitHub repository](https://github.com/aws-samples/sagemaker-genai-hosting-examples/tree/main/05-agents/strands/sagemaker-open-ai-compatible-endpoints-agentcore-runtime). The following diagram illustrates this architecture.\n\n## Prerequisites\n\nYou must have the following prerequisites to follow along with this post.\n\n- An AWS account with permissions for Amazon SageMaker AI, Amazon Bedrock, and AgentCore.\n\n`pip install sagemaker-core openai httpx strands-agents[otel] yfinance pydantic bedrock-agentcore`\n\n.\n\n- An AWS Identity and Access Management (IAM) role with\n`sagemaker:InvokeEndpoint`\n\nand`sagemaker:CallWithBearerToken`\n\n. - Bedrock model access for Claude Haiku 4.5 and Claude Sonnet 4.6.\n- Python 3.12+.\n\n## Step 1: Deploy Qwen 3.5 9B on SageMaker AI\n\nDeploy Qwen 3.5 9B using the vLLM Deep Learning Container (DLC), image `vllm:0.22.1-gpu-py312-cu130`\n\n, on `ml.g6e.2xlarge`\n\n.\n\n## Step 2: Build the multi-agent system\n\nThe OpenAI-compatible API of SageMaker AI expects a bearer token. Tokens expire, so for any long-running agent session you need a way to refresh them on every request. Set up auto-refreshing bearer tokens with an `httpx.Auth`\n\nsubclass:\n\nBuild using Strands Agents’ *agents as tools* pattern with fresh agent instances per invocation.\n\n## Step 3: Deploy to Amazon Bedrock AgentCore runtime\n\nDeploy using the `bedrock-agentcore-starter-toolkit`\n\n. See [deploy_agentcore.ipynb](https://github.com/aws-samples/sagemaker-genai-hosting-examples/blob/main/05-agents/strands/sagemaker-open-ai-compatible-endpoints-agentcore-runtime/Lab3/deploy_agentcore.ipynb) for the full deployment notebook.\n\n## Configure observability for SageMaker endpoints\n\nAmazon Bedrock AgentCore runtime instruments your agents with OpenTelemetry automatically, but that instrumentation doesn’t extend equally to every model provider. Before you can monitor cost and latency for the Qwen model on Amazon SageMaker AI, you must understand where the default instrumentation falls short and how to close that gap.\n\n### The challenge: Invisible token usage\n\nAmazon Bedrock AgentCore runtime automatically instruments agents using OpenTelemetry. However, there is a critical gap:\n\n**Amazon Bedrock model calls** get full generative AI spans with token counts automatically. No extra work is needed.**Amazon SageMaker OpenAI-compatible endpoints**(through Strands`OpenAIModel`\n\n) don’t get automatic token telemetry. The instrumentation doesn’t recognize them as generative AI calls.\n\nThis means tokens consumed by the financial analysis agent calling Qwen 3.5 9B on Amazon SageMaker are completely invisible in traces. You cannot monitor cost, detect regressions, or debug latency.\n\n**Root cause:** Strands’ OTEL integration emits spans for tool calls and agent lifecycle events, but it doesn’t emit `gen_ai.chat`\n\nspans with token attributes for the `OpenAIModel`\n\nprovider. The auto-instrumentation of AgentCore only recognizes Amazon Bedrock model inference calls (made through `boto3`\n\n) as generative AI operations.\n\n### The solution: Custom OpenTelemetry spans\n\nManually emit a `gen_ai.chat`\n\nspan that wraps the Amazon SageMaker agent invocation and extracts token usage from Strands’ internal `AgentResult.metrics.accumulated_usage`\n\n:\n\n**Key detail:** Strands tracks token usage internally with keys `inputTokens`\n\n, `outputTokens`\n\n, and `totalTokens`\n\n. This dict is populated only if the model provider returns usage data.\n\n### Why stream_options is mandatory for vLLM\n\nBy default, vLLM doesn’t include a usage chunk in streaming responses. Strands receives text chunks but never a final usage object. As a result, `accumulated_usage`\n\nstays at zero. Adding `stream_options: {\"include_usage\": True}`\n\ntells vLLM to send an extra final chunk with token counts:\n\nWithout this parameter, your `gen_ai.chat`\n\nspans report 0 tokens. This defeats the purpose of the custom span.\n\n### Step-by-step configuration\n\n**Turn on Amazon CloudWatch Transaction Search**(one-time per account or Region):** Install Strands with OTEL extras:**`strands-agents[otel]>=1.0.0`\n\n.**Set**`AGENT_OBSERVABILITY_ENABLED=true`\n\nin your code or env vars.**Use**`opentelemetry-instrument`\n\nas the container CMD.**Add**`stream_options: {\"include_usage\": True}`\n\nto`OpenAIModel`\n\nparams.**Create custom**`gen_ai.chat`\n\nspan wrapping the SageMaker agent call.\n\n### Example trace output\n\n### Agent trajectory on Bedrock AgentCore Observability dashboard\n\nThis trace view shows the `gen_ai.chat`\n\nspan for the Amazon SageMaker AI hosted Qwen model alongside the automatically instrumented Amazon Bedrock AgentCore spans, with token counts now visible for both. Building this end-to-end observability surfaced several implementation details worth calling out.\n\n### Key learnings\n\n**Amazon Bedrock AgentCore auto-instruments Bedrock calls**– No extra work for Claude or Amazon Nova.** SageMaker OpenAI endpoints need manual spans**– Strands doesn’t emit`gen_ai.chat`\n\nspans for`OpenAIModel`\n\n.**Token usage requires stream_options**– vLLM doesn’t send usage in streaming by default.** Use result.metrics.accumulated_usage**– Keys:`inputTokens`\n\n,`outputTokens`\n\n,`totalTokens`\n\n.**AWS X-Ray sampling rate matters**– Default 1 percent drops most traces. Use 100 percent during development.** Fresh agent instances per request**– Singletons cause concurrent invocation errors.\n\n### Extending the pattern\n\nThis architecture is composable. A few directions to explore:\n\n**Swap in fine-tuned models**: Point`SM_VLLM_MODEL`\n\nto your fine-tuned checkpoint on Amazon Simple Storage Service (Amazon S3). The auth layer, OTEL spans, and AgentCore deployment stay unchanged.**A/B test with inference components:** Deploy base and fine-tuned variants on the same Amazon SageMaker endpoint. Add a variant attribute to your OTEL span to compare quality in traces.**Cost-aware routing:** Check query complexity before dispatch. Route simple lookups to Haiku on Amazon Bedrock. Reserve the Amazon SageMaker GPU endpoint for multi-step reasoning tasks.\n\n## Cleaning up\n\nTo avoid incurring future charges, delete the resources:\n\n## Conclusion\n\nIn this post, we showed how to connect a self-hosted model on Amazon SageMaker AI to Amazon Bedrock AgentCore runtime, and critically, how to get full token-level observability from Amazon SageMaker endpoints that Strands Agents doesn’t instrument by default.\n\n`httpx.Auth`\n\n+`generate_token()`\n\n+`AsyncOpenAI`\n\n– Production-ready SageMaker authentication inside AgentCore.- Custom\n`gen_ai.chat`\n\nOTEL span +`stream_options: {\"include_usage\": True}`\n\n– Full token visibility for Amazon SageMaker endpoints. `result.metrics.accumulated_usage`\n\n– The Strands API for extracting token counts.\n\nTo get started, clone the [accompanying repository](https://github.com/aws-samples/sagemaker-genai-hosting-examples/tree/main/05-agents/strands/sagemaker-open-ai-compatible-endpoints-agentcore-runtime) and see [OBSERVABILITY.md](https://github.com/aws-samples/sagemaker-genai-hosting-examples/blob/main/05-agents/strands/sagemaker-open-ai-compatible-endpoints-agentcore-runtime/Lab3/OBSERVABILITY.md) for the complete reference.\n\n## Related resources\n\n[OpenAI-compatible API for SageMaker AI](/blogs/machine-learning/announcing-openai-compatible-api-support-for-amazon-sagemaker-ai-endpoints/)[Strands Agents — agents as tools](https://strandsagents.com/latest/documentation/docs/user-guide/concepts/multi-agent/agents-as-tools/)[Amazon Bedrock AgentCore Observability](https://docs.aws.amazon.com/bedrock-agentcore/latest/devguide/observability-get-started.html)[OpenTelemetry generative AI semantic conventions](https://opentelemetry.io/docs/specs/semconv/gen-ai/)", "url": "https://wpnews.pro/news/building-agentic-workflows-with-sagemaker-ai-and-bedrock-agentcore", "canonical_source": "https://aws.amazon.com/blogs/machine-learning/building-agentic-workflows-with-sagemaker-ai-and-bedrock-agentcore/", "published_at": "2026-08-14 15:58:44+00:00", "updated_at": "2026-08-14 16:11:49.260738+00:00", "lang": "en", "topics": ["artificial-intelligence", "machine-learning", "large-language-models", "ai-infrastructure", "ai-agents"], "entities": ["Amazon Web Services", "Amazon SageMaker AI", "Amazon Bedrock AgentCore", "Qwen 3.5 9B", "Strands Agents", "Claude Haiku 4.5", "Claude Sonnet 4.6", "GitHub"], "alternates": {"html": "https://wpnews.pro/news/building-agentic-workflows-with-sagemaker-ai-and-bedrock-agentcore", "markdown": "https://wpnews.pro/news/building-agentic-workflows-with-sagemaker-ai-and-bedrock-agentcore.md", "text": "https://wpnews.pro/news/building-agentic-workflows-with-sagemaker-ai-and-bedrock-agentcore.txt", "jsonld": "https://wpnews.pro/news/building-agentic-workflows-with-sagemaker-ai-and-bedrock-agentcore.jsonld"}}