How to deploy a Pre-Trained model using AWS SageMaker Notebook Instance?

Question

I have a pre-trained model which I am loading in AWS SageMaker Notebook Instance from S3 Bucket and upon providing a test image for prediction from S3 bucket it gives me the accurate results as required. I want to deploy it so that I can have an endpoint which I can further integrate with AWS Lambda Function and AWS API GateWay so that I can use the model with real time application. Any idea how can I deploy the model from AWS Sagemaker Notebook Instance and get its endpoint? Code inside the .ipynb file is given below for reference.

import boto3
import pandas as pd
import sagemaker
#from sagemaker import get_execution_role
from skimage.io import imread
from skimage.transform import resize
import numpy as np
from keras.models import load_model
import os
import time
import json
#role = get_execution_role()
role = sagemaker.get_execution_role()

bucketname = 'bucket' # bucket where the model is hosted
filename = 'test_model.h5' # name of the model
s3 = boto3.resource('s3')
image= s3.Bucket(bucketname).download_file(filename, 'test_model_new.h5')
model= 'test_model_new.h5'

model = load_model(model)

bucketname = 'bucket' # name of the bucket where the test image is hosted
filename = 'folder/image.png' # prefix
s3 = boto3.resource('s3')
file= s3.Bucket(bucketname).download_file(filename, 'image.png')
file_name='image.png'

test=np.array([resize(imread(file_name), (137, 310, 3))])

test_predict = model.predict(test)

print ((test_predict > 0.5).astype(np.int))

Muhammad Arsalan Hassan · Accepted Answer · 2024-10-20 11:44:41Z

4

Here is the solution that worked for me. Simply follow the following steps.

1 - Load your model in the SageMaker's jupyter environment with the help of

from keras.models import load_model

model = load_model (<Your Model name goes here>) #In my case it's model.h5

2 - Now that the model is loaded convert it into the protobuf format that is required by AWS with the help of

def convert_h5_to_aws(loaded_model):

from tensorflow.python.saved_model import builder
from tensorflow.python.saved_model.signature_def_utils import predict_signature_def
from tensorflow.python.saved_model import tag_constants

model_version = '1'
export_dir = 'export/Servo/' + model_version
# Build the Protocol Buffer SavedModel at 'export_dir'
builder = builder.SavedModelBuilder(export_dir)
# Create prediction signature to be used by TensorFlow Serving Predict API
signature = predict_signature_def(
    inputs={"inputs": loaded_model.input}, outputs={"score": loaded_model.output})
from keras import backend as K

with K.get_session() as sess:
    # Save the meta graph and variables
    builder.add_meta_graph_and_variables(
        sess=sess, tags=[tag_constants.SERVING], signature_def_map={"serving_default": signature})
    builder.save()
import tarfile
with tarfile.open('model.tar.gz', mode='w:gz') as archive:
    archive.add('export', recursive=True)
import sagemaker

sagemaker_session = sagemaker.Session()
inputs = sagemaker_session.upload_data(path='model.tar.gz', key_prefix='model')
convert_h5_to_aws(model):

3 - And now you can deploy your model with the help of

!touch train.py
from sagemaker.tensorflow.model import TensorFlowModel
sagemaker_model = TensorFlowModel(model_data = 's3://' + sagemaker_session.default_bucket() + '/model/model.tar.gz',
                                  role = role,
                                  framework_version = '1.15.2',
                                  entry_point = 'train.py')
%%timelog
predictor = sagemaker_model.deploy(initial_instance_count=1,
                                   instance_type='ml.m4.xlarge')

This will generate the endpoint which can be seen in the Inference section of the Amazon SageMaker and with the help of that endpoint you can now make predictions from the jupyter notebook as well as from web and mobile applications. This YouTube tutorial by Liam and AWS blog by Priya helped me a lot.

edited Oct 20, 2024 at 11:44

answered Mar 20, 2021 at 10:03

Muhammad Arsalan Hassan

6276 silver badges25 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Baenka Over a year ago

I have a resnet pytorch model that I have trained locally. I want to deploy the model using sagemaker endpoint. I have followed the steps at sagemaker.readthedocs.io/en/stable/frameworks/pytorch/…. However, I get an error that my inference.py file is missing. I have created a model folder as specified, converted it to a .tar.gz locally and have uploaded it to s3 bucket. I pass s3 location to the Pytorch constructor. I do not know what I'm missing. Kindly assist.

Muhammad Arsalan Hassan Over a year ago

Probably you haven't created any inference.py file. If you look at the 3rd point in my answer I am creating a train.py file using !touch train.py. This train.py is used for inference. In your case I have looked at the docs via the link that you've provided. There they've used the inference.py file as an entry point but they haven't mentioned where they created it. My suggestion would be to create that file yourself just before you import the execution role.

Collectives™ on Stack Overflow

How to deploy a Pre-Trained model using AWS SageMaker Notebook Instance?

1 Answer 1

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related