# Production MLOps at GS Caltex with Amazon SageMaker Canvas, part 1

> Source: <https://aws.amazon.com/blogs/machine-learning/production-mlops-at-gs-caltex-with-amazon-sagemaker-canvas-part-1/>
> Published: 2026-11-13 23:22:09+00:00

## [Artificial Intelligence](https://aws.amazon.com/blogs/machine-learning/)

# Production MLOps at GS Caltex with Amazon SageMaker Canvas, part 1

[GS Caltex](https://www.gscaltex.com/en) uses [Amazon SageMaker Canvas](https://aws.amazon.com/sagemaker/canvas/) and [Amazon SageMaker Pipelines](https://aws.amazon.com/sagemaker/ai/pipelines/) to connect process engineers’ models to production workflows. The South Korean energy and chemical company divides model development and operations between engineers with complementary skills.

Process engineers build and validate models in SageMaker Canvas without writing code, drawing on their process knowledge. Machine learning (ML) engineers configure SageMaker Pipelines workflows for deployment and scheduled inference.

Moving assets from SageMaker Canvas into SageMaker Pipelines still required integration work. ML engineers had to set data paths for production, supply the model configuration, and connect preprocessing and inference jobs. GS Caltex automated this handoff while reusing the assets created in SageMaker Canvas.

GS Caltex reports that its ML engineers now deploy models within a few hours, compared with about a week before. The company runs batch inference every 30 minutes, reducing the need for continuously running endpoints. GS Caltex reports that this has meaningfully lowered its inference costs.

This post explains how GS Caltex connects SageMaker Canvas to SageMaker Pipelines and reuses validated preprocessing flows and models in production.

## Solution overview

The machine learning operations (MLOps) workflow has two paths: preparing a new model version for deployment, and running batch inference with the deployed model.

### From a validated model to a repeatable workflow

You can use SageMaker Canvas to prepare data visually, build models, and generate batch predictions. You can also register validated models in Amazon SageMaker Model Registry from SageMaker Canvas. GS Caltex built a separate handoff process to turn these assets into executable SageMaker Pipelines steps.

Process engineers understand the variables and data behind a process and assess whether a model’s predictions are useful. ML engineers turn the resulting assets into production jobs and manage model versions and execution history.

| **Area** | **What is prepared in SageMaker Canvas** | **What is managed in the production workflow** | 
| Data preparation | A preprocessing flow built with Data Wrangler | Preprocessing jobs in Amazon SageMaker Processing | 
| Model development | A trained model and evaluation results | Model versions and approval status in Model Registry | 
| Prediction | Validation of model predictions | Batch inference every 30 minutes using an approved model | 
| Operations | Development and validation of individual models | Step execution history, stored results, and failure handling | 

### Reference architecture

The following diagram shows how process engineers hand off SageMaker Canvas assets to ML engineers for deployment and scheduled inference. ML engineers use Amazon Simple Storage Service (Amazon S3) for preprocessing flows and Model Registry for model versions, then schedule pipeline runs through Amazon EventBridge.

ML engineers receive the preprocessing flow and model from SageMaker Canvas and review the data formats and prediction results. After that review, they approve the model in Model Registry and deploy it. This preparation takes place when a new model version is deployed. Subsequent inference runs reuse the deployed model.

Amazon EventBridge starts each scheduled run. SageMaker Pipelines prepares the input data from Amazon Simple Storage Service (Amazon S3), runs preprocessing and [SageMaker Batch Transform](https://docs.aws.amazon.com/sagemaker/latest/dg/batch-transform.html), and stores the results in S3. ML engineers can inspect step status and Amazon CloudWatch logs to monitor the workflow.

The production pipeline reuses the preprocessing logic and trained model created in SageMaker Canvas. ML engineers configure the input and output paths, set the order of execution, and write the code that connects approval and scheduling to the workflow.

## Prerequisites

Before you begin, prepare the following:

- A [model built and evaluated in SageMaker Canvas](https://docs.aws.amazon.com/sagemaker/latest/dg/canvas-custom-models.html) , along with its[Data Wrangler preprocessing flow](https://docs.aws.amazon.com/sagemaker/latest/dg/canvas-data-prep.html)
- CSV data for inference and an S3 bucket to store the files.
- A [SageMaker execution role](https://docs.aws.amazon.com/sagemaker/latest/dg/sagemaker-roles.html) and an[Amazon EventBridge role with permission to start the pipeline](https://docs.aws.amazon.com/sagemaker/latest/dg/pipeline-eventbridge.html)
- A Python environment with Boto3 and SageMaker Python SDK 2.x installed.

The full reference implementation is available in the [GitHub repository](https://github.com/aws-samples/sample-canvas-to-pipelines-mlops). The code examples in this post focus on the connections between components. The repository uses a sample loan classification model, so adjust the configuration to match your model’s input and output formats.

## Technical implementation

The following steps cover preprocessing reuse, model approval and deployment, pipeline construction, and scheduled inference.

### 1. Run the SageMaker Canvas preprocessing flow with SageMaker Processing

A preprocessing flow created with Data Wrangler in SageMaker Canvas records the data sources and transformation steps. To set up the reference implementation, [export data to Amazon S3](https://docs.aws.amazon.com/sagemaker/latest/dg/canvas-export-data.html) using SageMaker Processing, then retrieve the job settings and the `.flow` file used by that job.

In the export settings, turn off **Auto job configuration** and select **SageMaker Processing**. After the export starts, open the Processing job details on the Amazon SageMaker AI console. Record the container image URI, the S3 URI of the input named `flow`, the execution role, and the output settings. Use these values to configure the pipeline step that runs the `.flow` file.

The SageMaker execution role must have permissions to read the required S3 data, write the results, and access the container image. Choose the instance type and count based on the volume of data you need to process.

Keep the preprocessing logic validated by the process engineer, and adjust the data sources, paths, and other settings for the production environment. Check the container image and output node against the original job configuration.

| **Item to check** | **What to do** | 
| Input location | Check the S3 URI referenced by the data source in the `.flow` file. Uploading raw data to another location does not update this reference. | 
| Container image | Use the image from the Processing job created during the S3 export, and pin a version that reproduces the validated transformations. | 
| Output ID | Use the output node and output name from the exported configuration. If the required format is `{node_id}.{output_name}` , match it exactly. | 
| Learned transformation values | Preserve values learned during training, such as scaling and encoding parameters. Do not refit them on each inference batch. | 
| Validation | Compare the columns, data types, and transformed values produced by SageMaker Canvas and Processing for the same input. | 

Some `.flow` files contain execution metadata such as `internal_metadata.dw_job.container_uri`. Check the actual Processing job settings rather than assuming that every export contains this field. If you move the workflow to another AWS Region, verify that the container image is available there.

The reference implementation separates intermediate and final outputs by execution under `runs/<execution-id>/`. Its `PrepareInput` step writes data to a fixed S3 location referenced by the `.flow` file. Prevent executions that share that input path from overlapping.

To isolate input data for each execution as well, add a preparation step that updates the source URI in a copy of the `.flow` file. Declaring a SageMaker Pipelines parameter does not automatically replace a URI inside the file.

### 2. Handle differences between training and inference inputs

The target column is one difference between training and inference data. A flow built with training data can contain transformations that expect the target column in the input schema. Inference data does not contain that known outcome.

The reference implementation adds a `PrepareInput` step before preprocessing. If the target column is missing, this step adds it with a placeholder value. After preprocessing, `DropTargetColumn` removes the column. Match the placeholder’s data type and column position to the training schema, and pass only the expected features to the model in their original order.

Use this approach only when the placeholder target does not affect feature values. A flow that filters rows by the target, derives features from it, or applies target encoding needs additional treatment. In those cases, separate the transformations and learned state required for inference.

Validation should cover column order and count, data types, missing-value handling, and row count. After removing the target, check that the remaining columns match the feature order used during training.

### 3. Register, approve, and deploy the model

Register the validated SageMaker Canvas model version in [Model Registry](https://docs.aws.amazon.com/sagemaker/latest/dg/canvas-register-model.html). The registered version can include the model files, inference configuration, and evaluation results. ML engineers review this information before changing the approval status.

The model file, such as `model.tar.gz`, must be used with the correct inference container and environment settings. A model package in Model Registry groups this information together. In this batch architecture, deployment creates a SageMaker model resource from an approved package for Batch Transform to reference.

The following example checks a package that has already been approved, then creates the model resource. Supply `model_package_arn`, `model_name`, and `sagemaker_role_arn` during deployment. This example uses a unique model name for each version.

Model Registry records approval status, but it does not enforce an approval check on every inference request. Check that status in the deployment code and retain the selected package ARN and model name in the pipeline configuration. If approval is revoked, update the schedules and execution configurations that use the model.

Registration and deployment take place when a model version is released. The pipeline that runs every 30 minutes creates Batch Transform jobs using that model, so it does not need to register the model again on each run.

### 4. Connect preprocessing, inference, and result assembly

The inference pipeline has five steps.

| **Step** | **Function** | **Input and output** | 
| `PrepareInput` | Add a placeholder target column if needed | New inference data -> data with the target column supplied | 
| `PreprocessNewData` | Run the SageMaker Canvas preprocessing flow | Prepared input and `.flow` -> transformed data | 
| `DropTargetColumn` | Remove the target column and CSV header | Transformed data -> feature CSV without a header | 
| `BatchPredict` | Run batch inference with the approved model | Feature CSV -> prediction files | 
| `AssembleResults` | Combine the preprocessed data with predictions | Preprocessed data and prediction files -> final result CSV | 

Configure each job before connecting the steps. In the following example, `prepare_args`, `preprocess_args`, `drop_target_args`, and `assemble_args` hold the step arguments returned by `run()` calls on processors that use a `PipelineSession`. The processors and `transformer` share the `pipeline_session` object. Configure the `transformer` with the previously deployed `model_name`.