Migrate your prompts to new models and optimize them on Amazon Bedrock

wpnews.pro

The problem: Prompt engineering at scale is still painful #

Migrating prompts to new models on Amazon Bedrock, or optimizing them for your current model, is still one of the most manual parts of building a generative AI application. Say you have built a deployed generative AI application. It works. Your prompts are tuned, your outputs are consistent, and your users are happy. Then a new model becomes available on Amazon Bedrock that is faster, cheaper, and more capable, and you must decide whether to migrate.

The decision should be straightforward, but it rarely is.

Customers spend days to weeks optimizing prompts and re-evaluating responses when they migrate to a new model. The same effort applies when they try to improve performance on their current model. The cycle looks the same every time. Rewrite the prompt, run it against test cases, compare results, tweak, and repeat. Now multiply that work by every prompt template in production and every model candidate worth evaluating. The result is a problem that scales with your ambition.

Today, Amazon Bedrock introduces Advanced Prompt Optimization, a tool that optimizes prompts for up to 5 models on Bedrock while comparing original and optimized performance. In this post, we show you how to use Amazon Bedrock Advanced Prompt Optimization to migrate and optimize prompts across multiple models in a single job. This approach replaces days of manual iteration with a guided, metrics-driven workflow.

The challenge: How this breaks the generative AI software development lifecycle #

Prompt migration and optimization sit at a critical chokepoint in the generative AI development lifecycle. When this step is slow or manual, you feel it in 4 areas:

Model lock-in. Teams avoid migration because the time and effort of re-tuning is too high, even when newer models would reduce latency and inference cost.Underperformance. When you write prompts for one model, you don’t use the full capability of another.Regression blindness. Without systematic evaluation against ground truth, you can’t distinguish “different” from “worse.” Teams ship degraded outputs without knowing it.Slow iteration cycles. Every prompt change requires manual A/B testing, which keeps engineers in evaluation loops instead of building features.

You need a prompt optimizer with built-in evaluations that test multiple models at once. It should also let you guide how prompts and responses change, grounding the optimization in real use cases and data.

The architecture: Metric-driven feedback loop #

Amazon Bedrock Advanced Prompt Optimization works backward from your evaluation metric. Instead of rewriting prompts heuristically, the system operates in a reinforcement learning-style feedback loop, without changing model weights:

You provide a prompt template, example user inputs (text or multimodal), optional ground truth answers, and an evaluation metric.
The optimizer sends your template and inputs to your chosen inference models. It evaluates responses against your metric and then rewrites the prompt.
The loop repeats (evaluate, rewrite, evaluate), iteratively optimizing your prompt against your evaluation metric.
The output is the original and optimized prompts. For each, you get evaluation scores, time to first token (TTFT) per sample, and inference cost estimates at on-demand pricing.

This architecture is model-agnostic. You can use your choice of model on Amazon Bedrock. A single job can target up to 5 models at once. You get an optimized prompt for each candidate and a direct comparison of quality, latency, and cost, all from one submission.

The following diagram shows how these pieces fit together.

What is TTFT? TTFT (time-to-first-token) measures how quickly the model starts responding: the time from sending a request to receiving the first token. It’s a useful proxy for perceived latency, like a chatbot beginning to type.

TTFT captures when the response starts, so pair it with output-token counts for the full latency picture. Since it also reflects real-time infrastructure conditions, read single-run values as a directional signal.

Advanced Prompt Optimization optimizes for your quality metric (the score) and reports TTFT and cost alongside every result. This gives you a complete view of quality, latency, and cost to choose the right prompt with confidence.

Solution: Amazon Bedrock Advanced Prompt Optimization #

Amazon Bedrock introduces Advanced Prompt Optimization, a tool that optimizes prompts for up to 5 models on Bedrock while comparing original and optimized performance. You can use it in two ways:

Model migration. Select your current model as a baseline plus up to 4 candidates. You get optimized prompts for each and a side-by-side comparison.Same-model improvement. Select only your current model. You get a better prompt without changing anything else.

Three evaluation modes

You choose one evaluation method per prompt template, and each method serves a different use case. The following table summarizes when to use each one.

If you omit all evaluation fields, the system uses a built-in default that combines accuracy, answer completeness, and writing style scoring. We recommend you define your own evaluation metric for the best results. If you have more than one objective to optimize for, you can create a composite metric with weights for each of your criteria in the scoring algorithm. Your metric output must be a single number.

Multimodal support

Advanced Prompt Optimization supports multimodal inputs: PNG, JPG, JPEG, GIF, WebP, and PDF files referenced through Amazon Simple Storage Service (Amazon S3) URIs. You can optimize prompts for document analysis, image classification, visual question answering, and other tasks that combine text with visual input.

Getting started: Console #

This section walks you through the console flow step by step: you create a job, select models, upload your input dataset, submit the optimization, and read the results. The Advanced Prompt Optimization landing page lists your jobs and their status and provides a button to start a new one, as shown in the following screenshot.

Step 1: Create a prompt optimization job

In the Amazon Bedrock console, in the left navigation pane, choose Advanced Prompt Optimization. On the landing page, choose Create prompt optimization. The next screenshot shows the creation page.

Step 2: Select inference models

In the Model selection section, choose Add model. Select your current model as a baseline. To add candidates, choose Add model again and select up to 4 additional models. If you are not migrating, select only your current model.

Step 3: Prepare your input dataset (JSONL)

Your input file uses JSONL format, with one JSON object per line, and each line is one prompt template to optimize. For full details, see the technical documentation. The following snippet shows the core schema.

Then add one of the evaluation method fields. For more details and overall guidance, see the technical documentation. The following snippet shows the AWS Lambda metric option.

The following snippet shows the LLM-as-a-Judge option. Authoring your judge rubric.In yourcustomLLMJPrompt

, reference 3 placeholders with double curly braces:`{{prompt}}`

(the fully rendered prompt, which is the prompt template combined with the evaluation sample),`{{response}}`

(the model output), and`{{referenceResponse}}`

(the ground truth). Advanced Prompt Optimization substitutes these values for each candidate and sample before sending the rubric to the judge model. For details, see[Use an LLM as a judge].

The following snippet shows the steering criteria option.

Step 4: Upload and submit

In the Prompt templates source section, choose Upload files directly from your local machine to upload your JSONL file. Alternatively, choose Import prompt templates from Amazon S3 and enter the S3 URI. In the S3 Output location section, enter your S3 output path. Choose Create optimization to submit the job.

Step 5: Interpret results

The results page shows 4 things: evaluation scores before and after optimization (per model), the optimized prompts ready to copy and deploy, TTFT per sample across models, and cost estimates based on on-demand inference pricing.

The following screenshots show the actual results of the examples we ran for this post: NESTFUL and XSum. We submitted them through the API and SDK (see the next section) using the datasets from the tutorial GitHub repo. Each job used 5 evaluation samples. The console renders the same results whether the job was created from the console or the API. You can use the repo as a reference for the expected output format and reproduce these views in your own account. Your results will vary based on your dataset, prompt complexity, and model choice. The summary view gives you the headline before and after scores. The following screenshot shows the NESTFUL run (function calling, Amazon Nova 2 Lite), where accuracy increased from 0.267 to 0.647.

The details view shows the original and optimized prompt side by side, along with per-sample metrics and the full token and latency breakdown, as shown in the following screenshot.

The following screenshot shows the summary view for the XSum run (summarization, Claude Haiku 4.5). The Quality Score increased from 0.550 to 0.743 while output tokens dropped from 51 to 28.

The following screenshot shows the corresponding details view for XSum.

Getting started: API / SDK (boto3) #

You can also drive the whole workflow with boto3. The following code creates a job that targets two candidate models.

Poll for completion

A job runs asynchronously, so poll its status until it reaches a terminal state. The following code checks every 60 seconds.

Download results

When the job completes, it automatically writes a JSONL results file to the Amazon S3 output location that you selected when you started the job. The following code uses boto3 to download it.

Full example: Steering criteria with multimodal input

The following example builds a steering-criteria dataset that pairs a question with a PDF document, writes it to JSONL, and uploads it to Amazon S3.

Key considerations #

Before you run a job, it helps to know the service limits and a few practices that keep optimization on track. The following table lists the per-job limits.

Limit | Value | | Templates per job | 10 | | Evaluation samples per template | 100 | | Models per job | 5 | | Text variables per template | 20 | | Multimodal files per sample | 100 | | Steering criteria per template | 5 |

A few practices matter more than the limits. Choose a single evaluation method per template, because combining steeringCriteria

with customLLMJConfig

or evaluationMetricLambdaArn

is not supported. When you write templates, use double curly braces ({{variable}} ) rather than single braces. Give each key in inputVariables

its own object, so `[{"var1": "val"}, {"var2": "val"}]`

rather than `[{"var1": "val", "var2": "val"}]`

. Finally, mix easy and hard examples in your dataset. An all-easy set won’t push the optimizer, and an all-hard set leaves it nothing to learn from.

If you use LLM-as-a-Judge, 3 judge models are currently available: `anthropic.claude-opus-4-6-v1`

, `anthropic.claude-sonnet-4-5-20250929-v1:0`

, and `anthropic.claude-sonnet-4-6`

. For model availability by Region, refer to Supported Regions, models, and quotas.

Results and metrics #

The following numbers come from 4 real Advanced Prompt Optimization jobs that we ran for this post in the AWS US East (N. Virginia) Region, each optimizing over 5 evaluation samples. The runs use small sample sets to illustrate the workflow rather than to establish a formal benchmark. Treat the figures as illustrative of the kinds of gains the optimizer surfaces, not as published benchmark results. You can reproduce them with the datasets in the tutorial GitHub repo.

A note on naming.XSum, NESTFUL, MM-VQA, and IFBench are theexample datasetsfrom the tutorial repo. They are not modes you choose. What you actually pick is the evaluation method (Lambda metric, LLM-as-a-Judge, or steering criteria), described in[Three evaluation modes]. You choose one per prompt template, based on your use case. These 4 datasets simply exercise all 3 methods so you can see each one in action.

The following table summarizes the quality gains across the 4 runs.

The next table shows what happens to latency and token counts alongside those quality gains.

Example | Score | TTFT | Input tokens | Output tokens | | XSum – before | 0.550 | 0.82 s | 720 | 51 | | XSum – after | 0.743 | 0.82 s | 1601 | 28 | | NESTFUL – before | 0.267 | 0.60 s | 1252 | 142 | | NESTFUL – after | 0.647 | 0.63 s | 3832 | 165 | | MM-VQA – before | 0.339 | 4.16 s | 30,998 | 142 | | MM-VQA – after | 0.777 | 4.02 s | 33,057 | 136 | | IFBench – before | 0.413 | 0.88 s | 175 | 440 | | IFBench – after | 0.890 | 0.54 s | 960 | 1359 |

Understanding the cost and latency trade-offs

Optimized prompts don’t just change quality. They change what you pay and how long you wait. Here’s how to read the numbers.

When you optimize a prompt, the token counts shift, and that is what drives cost. Bedrock bills per input and output token, with output tokens priced higher than input tokens. The clearest way to read these results is to compare input and output token counts before and after. The 4 runs land in different places, and you stay in control of where.

The optimizer maximizes your quality metric (the score) and reports tokens, TTFT, and cost so you have the full picture for each result. You can also shape the output directly: add a goal to your evaluation criteria that rewards shorter responses, and the optimizer favors prompts that produce them.

XSum delivered a 35% quality gain. The steering criteria (“exactly one sentence, 25 words or fewer”) produced much tighter answers, cutting output tokens from 51 to 28. The optimized prompt carries that guidance, so input tokens grew from 720 to 1,601. You spend more on input to get a higher-quality, more concise answer, a strong trade when short, accurate responses are the goal.

NESTFUL invested tokens for accuracy. The function-calling prompt grew from about 1.3K to about 3.8K input tokens as it added explicit decomposition rules and worked examples, and accuracy more than doubled (+143%). You spend more per call for a substantial quality improvement.

MM-VQA produced the largest quality gain for the smallest cost change. ROUGE-L rose from 0.339 to 0.777 (+129%) on document VQA, while input tokens grew only about 7% (30,998 to 33,057) and output tokens edged down from 142 to 136. Here the optimizer sharpened the prompt rather than lengthening it, so a large quality gain came with only a small cost change.

IFBench more than doubled quality. Constraint adherence climbed from 0.413 to 0.890 (+115%) on this LLM-as-a-Judge task. The optimized prompt spends more tokens because it spells out the constraints the model must satisfy. That makes it the most token-intensive of the 4 and the clearest case of investing tokens for quality.

The pattern across all 4 runs is consistent: optimization delivered large quality gains, and every optimized prompt used more total tokens. The increase ranged from about 7% more (MM-VQA) to several times more (IFBench). When output shrinks, as in XSum and MM-VQA, it softens the increase. The optimizer hands you the exact token counts and scores, so you can run the cost calculation for your own traffic and choose the trade-off that fits your workload. That quantified decision is the real advantage over tuning prompts by hand.

Reading the cost column. Bedrock bills per input and output token, and output tokens are generally more expensive than input tokens. A higher score that adds input tokens can be a great trade for a low-volume, high-stakes task, and a high-traffic endpoint benefits from keeping token growth in check. Because output tokens carry the higher price, you can add reducing output tokens as a goal in your evaluation criteria to keep inference cost down while you optimize for quality.

What the results tell you #

Compared to manual prompt tuning, Advanced Prompt Optimization gives you more than a higher score. Every run produces a quantified, comparable result across quality, latency, and cost in a single job. This turns a prompt that “feels better” into a change you can defend in a design review.

A few pitfalls are worth watching for as you read your own results:

Do the real cost calculation. A higher score that adds input tokens can still be the right call. Because output tokens cost more than input tokens, shorter optimized responses recover part of the added input cost. Compute the actual input-plus-output cost for your request volume so you weigh the quality gain against the true per-call cost.Treat TTFT as directional. TTFT on Bedrock reflects real-time network conditions, capacity, and time of day, and a longer input prompt tends to raise it. Use the reported TTFT as a rough signal, and confirm latency for your workload with your own repeated measurements.Choose your metric deliberately. The optimizer maximizes exactly what you measure, so define a metric that reflects real success. Give it signal to climb by mixing easy and hard examples. Cover as much of your expected production distribution as possible in your evaluation dataset so the optimized prompt generalizes.Use the right placeholders in LLM-as-a-Judge rubrics. Reference{{prompt}}

,`{{response}}`

, and`{{referenceResponse}}`

with double curly braces so the optimizer substitutes the right values and the judge scores each sample accurately.Match the evaluation method to your goal. Steering criteria guide the rewriter and feed a holistic score, which fits qualitative goals like tone and format. When you need a hard threshold, such as accuracy of 0.9 or higher, choose a Lambda metric for a precise, deterministic score.

Note: the results presented in this post are single-run results on tutorial datasets (5 samples each). Your results will vary with data distribution, prompt complexity, and model choice.

Conclusion #

With Amazon Bedrock Advanced Prompt Optimization, you remove the manual bottleneck from prompt migration and optimization. Instead of days of trial and error, you submit a job, define your success criteria, and get back optimized prompts with quantified improvements across multiple models at once.

The workflow stays the same across use cases. You might migrate to a newer model for cost savings, evaluate a multimodal model for document understanding, or extract more performance from your current setup. In each case you provide your template, your data, and your metric, and let the optimizer close the gap.

To learn more, see the following resources:

Amazon Bedrock.How Advanced Prompt Optimization works.API reference: CreateAdvancedPromptOptimizationJob.GitHub: tutorial notebooks for the three evaluation modes.Documentation: prepare your input dataset.

source & further reading

aws.amazon.com — original article Inference meta-monitoring for Amazon SageMaker AI endpoints with Amazon Quick Introducing explicit prompt caching for OpenAI GPT-5.6 models on Amazon Bedrock North Korea-linked hackers targeted major NPM packages