2

I am trying to use a XGBoost model in Sage Maker and use it to score for a large data stored in S3 using Batch Transform.

I build the model using existing Sagemaker Container as follows:

estimator = sagemaker.estimator.Estimator(image_name=container, 
                                          hyperparameters=hyperparameters,
                                          role=sagemaker.get_execution_role(),
                                          train_instance_count=1, 
                                          train_instance_type='ml.m5.2xlarge', 
                                          train_volume_size=5, # 5 GB 
                                          output_path=output_path,
                                          train_use_spot_instances=True,
                                          train_max_run=300,
                                          train_max_wait=600)

estimator.fit({'train': s3_input_train,'validation': s3_input_test})

The following code is used to do Batch Transform

 The location of the test dataset
batch_input = 's3://{}/{}/test/examples'.format(bucket, prefix)

# The location to store the results of the batch transform job
batch_output = 's3://{}/{}/batch-inference'.format(bucket, prefix)

transformer = xgb_model.transformer(instance_count=1, instance_type='ml.m4.xlarge', output_path=batch_output)

transformer.transform(data=batch_input, data_type='S3Prefix', content_type='text/csv', split_type='Line')

transformer.wait()

The above code works fine in Development environment (Jupyter notebook) when the model is built in Jupyter. However, I would like to deploy the model and call its endpoint for Batch Transform.

Most examples for SageMaker endpoint creation is for scoring on a single data and not for batch transform.

Can someone point to how to deploy and use the endpoints for Batch Transform in SageMaker? Thank you

1 Answer 1

1

The following link has an example of how to call a stored model in SageMaker to run Batch Transform job.

Batch Transform Reference

Sign up to request clarification or add additional context in comments.

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.