Unable to import module 'lambda function': libldap_r-2.4.so.2 cannot open shared objectfile: No such file or directory

Question

I've been dealing with this issue for the past few days now. I am trying to create a lambda layer with the following modules: cryptography, ldap3, and bonsai to use in my lambda_function that has a runtime of python3.11 and architecture type of x86_64. I have been using AWS CloudShell to create the lambda_layer zip file.

This is how I installed cryptography and ldap3:

-pip3.11 install \    
    --platform manylinux2014_x86_64 \
    --target=my-lambda-function \
    --implementation cp \
    --python-version 3.11.8 \
    --only-binary=:all: --upgrade \
    cryptography

-pip3.11 install \    
    --platform manylinux2014_x86_64 \
    --target=my-lambda-function \
    --implementation cp \
    --python-version 3.11.8 \
    --only-binary=:all: --upgrade \
    ldap3

These two modules work just fine I can call the module in my lambda function, the issue is with the bonsai module. Unfortunately, I can't install the bonsai module like this:

pip3.11 install \    
    --platform manylinux2014_x86_64 \
    --target=my-lambda-function \
    --implementation cp \
    --python-version 3.11.8 \
    --only-binary=:all: --upgrade \
    bonsai

Because I get the following error when I run this command:

ERROR: Could not find a version that satisfies the requirement bonsai (from versions:none)
ERROR: No matching distribution found for bonsai

Therefore this is how I am installing bonsai:

pip3.11 install bonsai --target=my-lambda-function

But when I try to call the bonsai module in my lambda function I get this error:

Unable to import module 'lambda function': libldap_r-2.4.so.2 cannot open shared objectfile: No such file or directory

Does anybody know how to resolve this issue, I've completely hit a brick wall. Any help is welcome!

jellycsc · Accepted Answer · 2024-03-07 20:20:50Z

0

It's much easier to build a lambda layer by using the Serverless Framework. I've built the layer for you. You can download it through this google drive link. This layer is for the x86_64 python3.11 runtime. Remember that you need to set the environment variable LD_LIBRARY_PATH to /opt/lib before using this layer.

Test code:

import json
import cryptography
import ldap3
import bonsai

def lambda_handler(event, context):
    # TODO implement
    print(cryptography.__version__)
    print(ldap3.__version__)
    print(bonsai.__version__)
    return {
        'statusCode': 200,
        'body': json.dumps('Hello from Lambda!')
    }

Test result:

Test Event Name
test

Response
{
  "statusCode": 200,
  "body": "\"Hello from Lambda!\""
}

Function Logs
START RequestId: 3cfbef06-dd11-4fba-bec8-48004ad2e959 Version: $LATEST
42.0.5
2.9.1
1.5.2
END RequestId: 3cfbef06-dd11-4fba-bec8-48004ad2e959
REPORT RequestId: 3cfbef06-dd11-4fba-bec8-48004ad2e959  Duration: 8.01 ms   Billed Duration: 9 ms   Memory Size: 128 MB Max Memory Used: 67 MB  Init Duration: 506.47 ms

Request ID
3cfbef06-dd11-4fba-bec8-48004ad2e959

If you don't trust my code, you can follow the steps below to build it yourself.

Step 1: Add a `requirements.txt`

cryptography==42.0.5
ldap3==2.9.1
bonsai==1.5.2

Step 2: Add a `Dockerfile`

FROM public.ecr.aws/sam/build-python3.11:latest-x86_64
RUN yum -y install openldap-devel

Step 3: Modify `serverless.yml`

service: example

plugins:
  - serverless-python-requirements

custom:
  pythonRequirements:
    dockerFile: Dockerfile
    dockerizePip: true
    layer: true
    dockerExtraFiles:
      - /lib64/libldap-2.4.so.2
      - /lib64/liblber-2.4.so.2
      - /lib64/libsasl2.so.3
      - /lib64/libssl3.so
      - /lib64/libsmime3.so
      - /lib64/libnss3.so
      - /lib64/libnssutil3.so

provider:
  name: aws
  runtime: python3.11
  stage: dev
  region: us-east-1
  profile: dummy

functions:
  dummy:
    handler: dummy.handler

Step 4: Build the layer

Simply do sls package -p pkg, then the layer zip named pythonRequirements.zip will be in pkg directory.

Step 5: Move `.so` files to a separate directory

Due to a bug in serverless-python-requirements, you need to unzip the pythonRequirements.zip, create a lib directory, move .so files into the new dir, and re-zip it.

answered Mar 7, 2024 at 20:20

jellycsc

12.5k2 gold badges20 silver badges41 bronze badges

Sign up to request clarification or add additional context in comments.

7 Comments

Lee Cavazos Over a year ago

Thank you so much for your reply! I went ahead and decided to follow the steps, when I run serverless package --package pkg I get this error: Error: docker build -f ./Dockerfile -t sls-py-reqs-custom . Exited with code 1 Do you know what is causing this issue, I followed all of your steps

jellycsc Over a year ago

Do you have docker installed?

Lee Cavazos Over a year ago

I found the issue, turns out I didn't have docker running. But I ran into another issue when I deployed the new lambda layer I got this error "Unable to import module 'lambda function': /lib64/libnssutil3.so: version 'NSSUTIL_3.82' not found (required by /opt/lib/libnss3.so)"

jellycsc Over a year ago

Remember that you need to set the environment variable LD_LIBRARY_PATH to /opt/lib before using this layer.

Lee Cavazos Over a year ago

Yup I added this line to the DockerFile: ENV LD_LIBRARY_PATH="/opt/lib". My apologies, I am fairly new to Docker, so I don't know if that is the proper way to set up the environment variable.

|

ddtraveller · Accepted Answer · 2024-03-07 19:31:24Z

0

You could try installing from the repo; pip install https://github.com/BonsaiAI/bonsai-python

You could build in CodeBuild and push the artifact to s3. This article talks about using CloudFormation to create the layer with a ref to an s3 artifact; https://www.linkedin.com/pulse/lambda-layer-creation-cloudformation-martin-macecek/

You could try to build inside a docker image with the same os as the lambda target os.

That .so file isn't available to the lambda os. I think that's the problem to work around somehow.

answered Mar 7, 2024 at 19:31

ddtraveller

1,24216 silver badges24 bronze badges

Collectives™ on Stack Overflow

Unable to import module 'lambda function': libldap_r-2.4.so.2 cannot open shared objectfile: No such file or directory

2 Answers 2

Step 1: Add a `requirements.txt`

Step 2: Add a `Dockerfile`

Step 3: Modify `serverless.yml`

Step 4: Build the layer

Step 5: Move `.so` files to a separate directory

7 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Step 1: Add a requirements.txt

Step 2: Add a Dockerfile

Step 3: Modify serverless.yml

Step 4: Build the layer

Step 5: Move .so files to a separate directory

7 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related

Step 1: Add a `requirements.txt`

Step 2: Add a `Dockerfile`

Step 3: Modify `serverless.yml`

Step 5: Move `.so` files to a separate directory