Uploading additional files for inference from local - Sagemaker deploy

Question

I'm attempting to deploy a local tensorflow model to a sagemaker endpoint. I want to include custom inference code to transform the input data. Given the model is trained and already located in an S3 bucket, I can run the following, which correctly deploys the model to an endpoint:

tensorflow_model = Model(
        model_data=saved_model,
        entry_point='src/development/document_matcher/inference.py',
        source_dir ='./src/development/document_matcher',      
        role=role, 
        framework_version=tf_framework_version
        )

The inference file is working correctly; however, I also need the vocabulary for the model to transform the incoming data correctly. This is what I'm trying to upload with source_dir.

My Directory structure is as follows:

- src
 - development
  -document_matcher
   - inference.py
   - total_vocab.pkl

I need to have total_vocab.pkl accessible to my inference script when it runs. However, inference.py is unable to find it. Am I misunderstanding how sagemaker works?

Marc Karp · Accepted Answer · 2022-02-23 01:22:27Z

0

According to the SageMaker TF container your total_vocab.pkl should be in /opt/ml/model/code

If it is not, seeing that your inference.py file is running I suggest adding some print statements to list directories.

answered Feb 23, 2022 at 1:22

Marc Karp

1,3246 silver badges6 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

bmasri · Accepted Answer · 2022-09-16 09:48:46Z

0

As stated in the documentation, 'If source_dir is specified, then entry_point must point to a file located at the root of source_dir'. Therefore, since your Model object already unzips the whole document_matcher in the same location as your model, you can simply put entry_point = inference.py. And your total_vocab.pkl object will be available in your inference.py code.

answered Sep 16, 2022 at 9:48

bmasri

4423 silver badges12 bronze badges

Collectives™ on Stack Overflow

Uploading additional files for inference from local - Sagemaker deploy

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related