AWS Lambda Docker Custom Python Library (Runtime.ImportModuleError)

Question

I'm trying to deploy a custom machine learning library on Lambda using a Lambda docker image.

The image is looking approximately as follows:

FROM public.ecr.aws/lambda/python:3.9

CMD mkdir -p /workspace
WORKDIR /workspace

# Copy necessary files (e.g. setup.py and library/)
COPY ...

# Install library and requirements
RUN pip3 install . --target "${LAMBDA_TASK_ROOT}"

# Copy lambda scripts
COPY ... "${LAMBDA_TASK_ROOT}/"

# CMD [ my_script.my_handler ]

Thus, it installs a local python package including dependencies to LAMBDA_TASK_ROOT (/var/task). The CMD (handler) is overriden in AWS, e.g. preprocessing.lambda_handler.

The container works fine for handlers that DO NOT use the custom python library (on AWS and locally). However, trying to use the custom library on AWS, it fails with Runtime.ImportModuleError claiming that "errorMessage": "Unable to import module 'MY_HANDLER': No module named 'MY_LIBRARY'".

Everything works when running the container locally with the runtime interface emulator. The file-level permissions should be ok as well.

Is there anything wrong with this approach?

David Muhr · Accepted Answer · 2022-06-24 07:39:48Z

2

Answering my own question here: There was no problem with the Docker container itself. The problem was that Lambda references docker image versions via their SHA, not the tag, thus update to the tag did not update the container of the functions. To update the container image, you have to use something like

aws lambda update-function-code --function-name $MY_FN --image-uri $MY_URI

answered Jun 24, 2022 at 7:39

David Muhr

413 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

AWS Lambda Docker Custom Python Library (Runtime.ImportModuleError)

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related