Launching UI for generative AI inference recommendations in Amazon SageMaker AI

Amazon SageMaker AI launched a UI for generative AI inference recommendations in SageMaker AI Studio, providing a low-code no-code experience for configuring, optimizing, and deploying models. The UI guides users through preset use-case profiles, visual comparisons, and one-click deployment, reducing the iteration cycle from weeks to minutes. This feature aims to help teams without deep infrastructure expertise quickly find production-ready configurations.

Artificial Intelligence https://aws.amazon.com/blogs/machine-learning/ Launching UI for generative AI inference recommendations in Amazon SageMaker AI Deploying generative AI models to production requires finding the right combination of instance type, serving container with settings, and optimization strategy. This process typically requires a long iteration cycle of optimization and manual benchmarking. In April 2026, Amazon SageMaker AI https://aws.amazon.com/sagemaker/ai launched this inference recommendations https://docs.aws.amazon.com/sagemaker/latest/dg/generative-ai-inference-recommendations.html , so customers can programmatically get data-driven, production-ready configurations through APIs. This feature compresses that cycle to minutes for common workloads, and a few hours for custom workloads. In this post, we introduce the UI for optimized generative AI inference recommendations https://aws.amazon.com/blogs/machine-learning/amazon-sagemaker-ai-now-supports-optimized-generative-ai-inference-recommendations/ in Amazon SageMaker AI Studio https://docs.aws.amazon.com/sagemaker/latest/dg/studio-updated.html , a low-code no-code LCNC experience. The API already gives you programmatic access to recommendations, but it assumes you know which parameters to set and how to interpret raw benchmark output. The UI removes that assumption. It guides you through preset use-case profiles, visual comparisons of results, and one-click deployment, so teams without deep infrastructure expertise can get a validated configuration on their own. With this launch, you can configure optimization jobs and select from preset use-case profiles. You can compare performance results and deploy recommended configurations to production endpoints. You can do this without writing a single line of code. Advanced users can still use the APIs for fine-grained configurations. Machine learning ML engineers can validate their next deployment while technical leaders evaluate cost-performance trade-offs. In both cases, the Studio experience speeds up the path from model selection to a configuration that is production-ready. Overview of the new Studio experience Amazon SageMaker AI Studio now includes the generative AI inference recommendations UI under Jobs , then Inference optimization . It provides a guided, end-to-end workflow that walks you through workload configuration, optimization, model selection, and deployment. Preset use-case profiles The workflow starts by defining the workload configuration. Rather than manually specifying token distributions and concurrency, you can select a preset use-case profile that captures a common traffic pattern. The Interact profile models chat-style workloads with short inputs and moderate outputs. Generate is tuned for content generation with longer outputs, and Summarize is optimized for the high input-to-output ratios of document summarization. When none of these match your workload, you can use the Custom profile to bring your own dataset and set concurrency, token lengths, and evaluation data yourself. The profile you select sets the benchmark parameters, so you don’t need to reason about them directly. Optimization goals Alongside a use-case profile, select an optimization goal that tells SageMaker AI what to prioritize. Minimize latency tunes for the lowest possible response time, which suits interactive applications where users wait on each token. Maximize throughput serves the highest number of tokens per second, a better fit for batch or high-volume workloads. Minimize cost finds the most cost-efficient configuration for the traffic you expect. The goal that you pick shapes both the techniques SageMaker AI applies and the way that it ranks the resulting recommendations. Supported model sources Now, you select the model that requires inference optimization. The UI meets that model wherever it already lives. You can pull a foundation model FM from the Amazon SageMaker JumpStart https://aws.amazon.com/sagemaker/ai/jumpstart/ catalog, which offers a variety of pre-trained models. Alternatively, you can point to your own model artifact on Amazon Simple Storage Service Amazon S3 https://aws.amazon.com/s3/ , reuse a registered package from your Model Registry, or select an existing SageMaker model from a previous deployment or training job. Walkthrough: Getting recommendations through the console In the following sections, we walk through a complete run, from creating a job to deploying the best configuration. Each step builds on the previous one, so you can read them in order the first time and then jump directly to the step you need on later runs. Prerequisites: You must have the following prerequisites to follow along: - An AWS account with an Amazon SageMaker AI Studio domain. AWS Identity and Access Management IAM https://aws.amazon.com/iam permissions for Amazon SageMaker AI inference optimization operations.- An IAM execution role with access to your model artifacts in Amazon S3. Pricing: No additional cost for generating recommendations. Standard compute costs apply for optimization jobs and endpoints provisioned during benchmarking. Step 1: Create an optimization job In the Studio left navigation pane, choose Jobs , then Inference optimization . Then choose Create or Optimize inference . With the job created, the next step is to tell SageMaker AI what you’re optimizing for. Figure 1: The Inference optimization landing page showing existing jobs Step 2: Configure your strategy Every business use case has different requirements: some have a very tight latency budget, while others have a stringent cost budget. Depending on your use case, select one of the use case profiles Interact , Generate , Summarize , or Custom and an optimization goal Minimize cost , Minimize latency , or Maximize throughput . Figure 2a: Strategy configuration showing use-case presets Generate, Interact, Summarize, Custom and optimization goals If you select Custom , additional fields appear. You can paste an S3 URI to your evaluation dataset in JSONL format, specify the number of simultaneous requests to simulate for concurrency, and set the maximum output token length. Figure 2b: Custom workflow showing dataset path, concurrency, and output tokens Step 3: Select your model and compute Use the Select button in the Model section to select the model that you want. A window appears with tabs for each model source. Figure 3a: Model selection modal showing the JumpStart tab with tabs for JumpStart, S3, Logged, and Deployable If you select S3, you must provide a name and your model artifact’s S3 URI path. Figure 3b: Model selection modal showing the S3 tab with tabs for JumpStart, S3, Logged, and Deployable With a model selected, you can either let SageMaker AI select the hardware for you or narrow the search to specific instance types. Selecting compute is optional. In step 2, if you select Minimize latency or Maximize throughput as your optimization goal, you can optionally expand the Compute section. From there, select specific instance types for example, ml.g6e.2xl or g7e.2xl for the job to evaluate. If you don’t specify, SageMaker AI selects compatible instances based on your model requirements. If you have reserved capacity through a flexible training plan, you can see it in the Source list. Note: SageMaker AI selects a recommended instance for the Minimize cost use case. Figure 3c: Compute section showing three instance types selected for the recommendations job to evaluate Your configuration is now complete, and you’re ready to launch the job and watch it run. Step 4: Launch and monitor Choose Optimize to submit the job. You land on the job Detail page, where you can watch the status transition from Running to Completed . Preset use cases like Interact can return recommendations quickly because they are matched against pre-validated configurations, while custom benchmarks might take longer depending on model size, instance types, and dataset. To stop a job while it’s running, use the Actions menu. Figure 4: A running optimization job showing the Actions button, Settings tab, and status Use the Detail page tabs to inspect your job. The Overview tab shows ranked recommendations with performance metrics, which appear after the job is complete. The Settings tab displays the configuration you submitted, including your use case, goal, and compute selections. The Details tab contains job metadata, timestamps, and output artifacts. Step 5: Review recommendations and deploy The Overview tab displays ranked inference packages. Each package shows the optimized configuration, performance metrics time to first token TTFT , inter-token latency ITL , throughput, and cost , and a Deploy button. Figure 5: Completed job showing optimized inference packages with Deploy buttons Choose Deploy on your preferred package. A window pre-fills the endpoint name and instance type. You can deploy to a new endpoint or update an existing one. SageMaker AI then runs three steps in sequence: it registers the optimized model, sets up the endpoint configuration, and provisions the endpoint. After the endpoint reaches In Service, you can invoke it immediately. Managing optimization jobs The Jobs, Inference optimization landing page gives you centralized management of your optimization jobs. You can use the search bar to filter jobs by name. To stop a running job, select the job, choose Actions, then choose Stop, and confirm. To delete old jobs, select a completed or failed job, choose Actions, then choose Delete, and confirm. You can also choose any job to view its Detail page, which includes Overview, Settings, and Details tabs. How it works The console experience and the API use the same recommendations and benchmarking infrastructure. SageMaker AI first analyzes your model architecture and memory requirements to reduce the configuration space and identify viable options. It then applies goal-aligned optimizations: speculative decoding for throughput or kernel tuning for latency, depending on your selected goal. SageMaker AI benchmarks every configuration on real GPU infrastructure using NVIDIA AIPerf https://github.com/ai-dynamo/aiperf with multi-run confidence intervals. Finally, it scores the results against your optimization goal and returns production-ready configurations, ranked from best to worst. The inference optimization job runs multiple steps depending on the optimization goal that you selected. For minimum cost, SageMaker AI creates an endpoint using the instance type it recommends and runs benchmark jobs against it. To check the In Service status while the job runs, open Deployments and then Endpoints. To review the benchmark runs, open Training and then Training Jobs, and select your job. For minimum latency, if the model architecture and instance type support it, SageMaker AI uses a kernel fine-tuned deployment. Otherwise, it creates one or more standard endpoints, each corresponding to an instance type. SageMaker AI then runs benchmark jobs as a training job for each combination. You can review the Endpoints and Training logs to monitor progress and troubleshoot issues. For maximum throughput, if the model architecture and instance type support speculative decoding https://aws.amazon.com/blogs/machine-learning/amazon-sagemaker-ai-introduces-eagle-based-adaptive-speculative-decoding-to-accelerate-generative-ai-inference/ , a training job runs first to train the draft model before the endpoints are deployed. SageMaker AI then runs benchmark jobs as a training job for each combination. As with the other goals, you can review the Endpoints and Training logs to monitor progress and troubleshoot issues. Figure 6a: In Service endpoints created by the inference optimization job for each instance type selected in Step 4 Figure 6b: In Progress training jobs shown for each benchmark test configuration Note: The optimization job automatically deletes the endpoints after it completes. Best practices There are a few best practices to keep in mind. You should re-run optimization jobs after fine-tuning or updating your model. You should also re-run them when new instance types become available in your AWS Region, when traffic patterns change significantly, or after serving container or framework upgrades. It’s also a good idea to run them at a regular cadence, for example every few weeks, because the Amazon SageMaker AI team continuously adds new findings. Conclusion The console experience for generative AI inference recommendations in Amazon SageMaker AI Studio makes data-driven infrastructure decisions accessible to teams across your organization. With preset use-case profiles, a guided optimization workflow, visual recommendation comparisons, and one-click deployment, you can move from model selection to a configuration validated against your workload faster. Start by creating your first optimization job in Amazon SageMaker AI Studio, and explore the documentation https://docs.aws.amazon.com/sagemaker/latest/dg/model-optimize.html for detailed guidance. Related resources - Blog: Amazon SageMaker AI now supports optimized generative AI inference recommendations https://aws.amazon.com/blogs/machine-learning/amazon-sagemaker-ai-now-supports-optimized-generative-ai-inference-recommendations/ API launch . - Documentation: Optimize model inference in Amazon SageMaker AI https://docs.aws.amazon.com/sagemaker/latest/dg/model-optimize.html . - Sample notebooks: Generative AI inference recommendations examples https://github.com/aws-samples/sagemaker-genai-hosting-examples/tree/main/03-features/gen-ai-inference-recommendations on the GitHub website.