Originally published on kuryzhev.cloud
Your internal Slack runbook bot is sending every incident prompt to OpenAI's servers — and your security team doesn't know yet. This is the exact situation I've seen play out on three separate teams in the past year. Someone wires up a quick chatbot for incident triage or runbook lookup, ships it, and six months later a SOC 2 audit uncovers that sensitive operational data has been leaving the AWS environment entirely. Running this Bedrock vs OpenAI API compliance checklist before you commit to either platform takes about 30 minutes and can save you a very uncomfortable conversation with your CISO.
Internal DevOps chatbots — Slack bots that query runbooks, incident triage assistants, on-call helpers that summarize recent alerts — sit at the intersection of three concerns that rarely get audited together: cost control, data residency, and IAM governance. Most teams default to OpenAI because the API is fast to integrate and the models are excellent. I get it. I've done it too. But the decision deserves more than "OpenAI is easy."
The specific pain point is this: when your chatbot processes incident data, it's often sending hostnames, internal service names, error messages, and occasionally credentials that leaked into logs. If that traffic goes to api.openai.com
, it's leaving your AWS account boundary. That matters enormously if your org has a BAA requirement, a SOC 2 Type II mandate, or operates under FedRAMP constraints.
AWS Bedrock solves this by keeping inference within your AWS account. The trade-off is more setup overhead and some real gotchas around model availability and logging configuration that aren't obvious from the docs. This checklist covers AWS-native teams running workloads in us-east-1
or us-west-2
, comparing Bedrock (Claude 3 Sonnet, Claude 3 Haiku, Titan) against OpenAI API (gpt-4o, gpt-3.5-turbo). It's not a general AI evaluation — it's an infrastructure decision framework for teams that already have an AWS footprint and need to make a defensible architecture choice.
For more context on securing AWS workloads in CI/CD pipelines, the kuryzhev.cloud DevOps archive has related patterns worth reviewing before you finalize your setup.
Run through each gate in order. If you hit a hard blocker, stop and document it before continuing.
aws_iam_role
with bedrock:InvokeModel
permission scoped to a specific model ARN. OpenAI requires storing an API key as a secret — ideally in AWS Secrets Manager or SSM Parameter Store. The question to audit: who in your org currently has secretsmanager:GetSecretValue
access? In practice, that permission is often over-granted. With Bedrock, you're working within IAM, which most teams already audit. With OpenAI, you're adding a secret rotation burden that frequently gets neglected.
aws bedrock list-foundation-models \
--region us-east-1 \
--output table \
--query "modelSummaries[?contains(modelId,'claude')].[modelId,modelLifecycle.status]"
Run this for every region you intend to deploy in. I've seen teams design around Claude 3 Sonnet only to discover it requires a manual access request with a 24-hour approval window — which is a problem when you're mid-sprint.ThrottlingException
; implement exponential backoff with jitter, base delay 1s, max 60s, max 5 attempts. OpenAI returns HTTP 429 with a Retry-After
header. The retry logic is different enough that you need explicit handling for both if you're building a fallback pattern. Also confirm you're using boto3>=1.28.57
— earlier versions don't include the bedrock-runtime
client and will fail silently in ways that are genuinely confusing to debug. These are the items that don't show up in the official getting-started guides but cause production incidents or compliance failures.
Bedrock Provisioned Throughput is not on-demand. If you accidentally provision a Model Unit instead of using on-demand inference, you're committing to approximately $21.50/hour for a minimum of one month. That's roughly $15,480/month. I have personally seen this happen when someone clicked the wrong option in the console during a proof-of-concept. Always verify your invocation type in Terraform before applying. Provisioned Throughput makes sense at scale — it does not make sense for a chatbot pilot.
Model Invocation Logging is off by default. This one fails audits regularly. Teams assume that because Bedrock is an AWS service, it logs everything like other AWS services do. It does not. You must explicitly enable it:
aws bedrock put-model-invocation-logging-configuration \
--logging-config '{
"cloudWatchConfig": {
"logGroupName": "/aws/bedrock/chatbot-invocations",
"roleArn": "arn:aws:iam::ACCOUNT_ID:role/bedrock-logging-role",
"largeDataDeliveryS3Config": {
"bucketName": "your-bedrock-logs-bucket",
"keyPrefix": "invocation-logs/"
}
},
"s3Config": {
"bucketName": "your-bedrock-logs-bucket",
"keyPrefix": "invocation-logs/"
},
"textDataDeliveryEnabled": true,
"imageDataDeliveryEnabled": false,
"embeddingDataDeliveryEnabled": false
}'
Note that textDataDeliveryEnabled: true
logs prompt and response content. Depending on your compliance posture, you may want this on for audit purposes — or off if prompts contain PII. Make the decision explicitly, not by omission.
Cross-region inference profiles break data residency assumptions. Bedrock inference profile IDs with a us.
prefix — for example, us.anthropic.claude-3-5-sonnet-20241022-v2:0
— indicate cross-region routing. Traffic may transit us-west-2 even if your primary region is us-east-1. If your data residency policy requires all inference to stay in a single region, use the standard model ID without the us.
prefix and verify you're not inadvertently calling a cross-region profile.
Watch out for the wrong boto3 client. Using the bedrock
client instead of bedrock-runtime
for inference calls throws UnrecognizedClientException
. The bedrock
client is for management APIs (listing models, managing provisioned throughput). The bedrock-runtime
client is for actual inference. This trips up almost everyone the first time.
Running this checklist manually once is fine for the initial decision. Enforcing it programmatically across your team is how you prevent drift.
Terraform module for least-privilege Bedrock IAM. The following module provisions the IAM role and CloudWatch log group your chatbot needs, scoped to a specific model ARN rather than a wildcard. See the Terraform AWS IAM role policy documentation for the full resource reference.
terraform {
required_providers {
aws = {
source = "hashicorp/aws"
version = "~> 5.40"
}
}
}
variable "chatbot_service_name" {
description = "Name tag for the internal chatbot service (e.g. slack-runbook-bot)"
type = string
}
variable "aws_region" {
default = "us-east-1"
}
resource "aws_iam_role" "bedrock_chatbot_role" {
name = "${var.chatbot_service_name}-bedrock-role"
assume_role_policy = jsonencode({
Version = "2012-10-17"
Statement = [{
Effect = "Allow"
Principal = { Service = "lambda.amazonaws.com" }
Action = "sts:AssumeRole"
}]
})
tags = {
Service = var.chatbot_service_name
ManagedBy = "terraform"
Compliance = "internal-chatbot"
}
}
resource "aws_iam_role_policy" "bedrock_invoke_policy" {
name = "bedrock-invoke-haiku-only"
role = aws_iam_role.bedrock_chatbot_role.id
policy = jsonencode({
Version = "2012-10-17"
Statement = [
{
Sid = "AllowBedrockInvokeHaiku"
Effect = "Allow"
Action = ["bedrock:InvokeModel", "bedrock:InvokeModelWithResponseStream"]
Resource = "arn:aws:bedrock:${var.aws_region}::foundation-model/anthropic.claude-3-haiku-20240307-v1:0"
},
{
Sid = "AllowBedrockInvocationLogsWrite"
Effect = "Allow"
Action = ["logs:CreateLogGroup", "logs:CreateLogStream", "logs:PutLogEvents"]
Resource = "arn:aws:logs:${var.aws_region}:*:log-group:/aws/bedrock/chatbot-invocations:*"
}
]
})
}
resource "aws_cloudwatch_log_group" "bedrock_invocation_logs" {
name = "/aws/bedrock/chatbot-invocations"
retention_in_days = 90 # Adjust per your compliance retention policy
tags = {
Service = var.chatbot_service_name
Compliance = "audit-trail"
}
}
output "chatbot_role_arn" {
description = "IAM role ARN to assign to chatbot Lambda/ECS task"
value = aws_iam_role.bedrock_chatbot_role.arn
}
Python cost estimator as a CI gate. Run this before any chatbot deployment to output a cost comparison table. Wire it into your CI pipeline as a pre-deployment step so token volume estimates are always reviewed. See the AWS Bedrock model IDs documentation for current pricing references per region.
#!/usr/bin/env python3
import argparse
PRICING = {
"bedrock_claude3_haiku": {"input": 0.25, "output": 1.25},
"bedrock_claude3_sonnet": {"input": 3.00, "output": 15.00},
"openai_gpt35_turbo": {"input": 0.50, "output": 1.50},
"openai_gpt4o": {"input": 5.00, "output": 15.00},
}
def calculate_cost(model: str, input_tokens: int, output_tokens: int) -> float:
p = PRICING[model]
return (input_tokens / 1_000_000 * p["input"]) + \
(output_tokens / 1_000_000 * p["output"])
def main():
parser = argparse.ArgumentParser(description="Bedrock vs OpenAI cost estimator")
parser.add_argument("--input-tokens", type=int, required=True,
help="Monthly input token volume")
parser.add_argument("--output-tokens", type=int, required=True,
help="Monthly output token volume")
args = parser.parse_args()
print(f"\n{'Model'::<30} {'Monthly Cost (USD)':>20}")
print("-" * 52)
results = {}
for model in PRICING:
cost = calculate_cost(model, args.input_tokens, args.output_tokens)
results[model] = cost
print(f"{model:<30} ${cost:>19.2f}")
cheapest = min(results, key=results.get)
print(f"\n✅ Cheapest: {cheapest} at ${results[cheapest]:.2f}/month")
pt_monthly = 21.50 * 24 * 30 # 1 Model Unit, 1 month minimum
if results.get("openai_gpt4o", 0) > pt_monthly:
print(f"⚠️ gpt-4o cost exceeds Bedrock PT floor (${pt_monthly:.0f}/mo) — evaluate Provisioned Throughput")
if __name__ == "__main__":
main()
Secret scanning enforcement. Add a pre-commit hook using trufflehog v3 to block OpenAI API key commits before they hit your repo. Install with brew install trufflehog
or pull the container: docker run ghcr.io/trufflesecurity/trufflehog:latest
. Run it as: trufflehog filesystem . --only-verified
. Pair this with an AWS Config managed rule — SECRETSMANAGER_SECRET_UNUSED
— to flag stale OpenAI keys that haven't been rotated in 90 days. Neither of these is a substitute for the other; you need both the pre-commit gate and the ongoing compliance check.
The Bedrock vs OpenAI API compliance decision isn't about which model is smarter. It's about whether your architecture can withstand a security review six months from now. Run the checklist, automate the gates, and make the decision explicitly rather than by default.