AWS Lambda Java to S3 - java.lang.ClassNotFoundException on com.amazonaws.services.s3.AmazonS3ClientBuilder

Question

I tried running the code below locally (as a maven project in IntelliJ) and it runs OK. When I try to run it as a Lambda function, I always get a java.lang.NoClassDefFoundError. Somehow, lambda makes it so the class cannot be found when I include the AmazonS3 line below.

package example;

import com.amazonaws.services.lambda.runtime.Context;
import com.amazonaws.services.lambda.runtime.RequestHandler;
import com.amazonaws.services.s3.AmazonS3;
import com.amazonaws.services.s3.AmazonS3ClientBuilder;

public class ParserLambda implements RequestHandler<Object, Object> {
    @Override
    public Object handleRequest(Object input, Context context) {
        String retval = "";
        try {
            AmazonS3 s3Client = AmazonS3ClientBuilder.standard().build();
        } catch (Exception ex) {
            return "{'status':'error', 'retval': '" + ex.getMessage() + "'}";
        }
        return "{'status':'done', 'retval': '" + retval + "'}";
    }
}

If I comment out the following line

//AmazonS3 s3Client = AmazonS3ClientBuilder.standard().build();

it gives: "{'status':'done', 'retval': ''}". This is a valid result.

When uncommented

AmazonS3 s3Client = AmazonS3ClientBuilder.standard().build();

I get:

{
  "errorMessage": "com/amazonaws/services/s3/AmazonS3ClientBuilder",
  "errorType": "java.lang.NoClassDefFoundError",
  "stackTrace": [
    "example.ParserLambda.handleRequest(ParserLambda.java:11)",
    "sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)",
    "sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)",
    "sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)",
    "java.lang.reflect.Method.invoke(Method.java:498)"
  ],
  "cause": {
    "errorMessage": "com.amazonaws.services.s3.AmazonS3ClientBuilder",
    "errorType": "java.lang.ClassNotFoundException",
    "stackTrace": [
      "java.net.URLClassLoader.findClass(URLClassLoader.java:381)",
      "java.lang.ClassLoader.loadClass(ClassLoader.java:424)",
      "java.lang.ClassLoader.loadClass(ClassLoader.java:357)",
      "example.ParserLambda.handleRequest(ParserLambda.java:11)",
      "sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)",
      "sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)",
      "sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)",
      "java.lang.reflect.Method.invoke(Method.java:498)"
    ]
  }
}

My pom file includes:

<dependency>
    <groupId>com.amazonaws</groupId>
    <artifactId>aws-java-sdk</artifactId>
    <version>1.11.286</version>
</dependency>

I have also tried:

AmazonS3 s3Client = AmazonS3ClientBuilder.defaultClient();
AmazonS3 s3Client = AmazonS3ClientBuilder.standard().withRegion(Regions.US_EAST_1).withForceGlobalBucketAccessEnabled(true).build();

but with no success.

bobasti · Accepted Answer · 2019-11-03 12:58:48Z

I encountered the same issue and solved it after a few different attempts.

Adjusting the following dependencies in my pom.xml was not enough (note the <scope>provided</scope> for aws-java-sdk dependency):

<!-- https://mvnrepository.com/artifact/com.amazonaws/aws-java-sdk -->
<dependency>
    <groupId>com.amazonaws</groupId>
    <artifactId>aws-java-sdk</artifactId>
    <version>1.11.664</version>
    <scope>provided</scope>
</dependency>
<!-- https://mvnrepository.com/artifact/com.amazonaws/aws-java-sdk-s3 -->
<dependency>
    <groupId>com.amazonaws</groupId>
    <artifactId>aws-java-sdk-s3</artifactId>
    <version>1.11.664</version>
</dependency>

Turns out I was missing the following plugin in pom.xml:

<build>
  <plugins>
    <plugin>
      <groupId>org.apache.maven.plugins</groupId>
      <artifactId>maven-shade-plugin</artifactId>
      <version>3.2.1</version>
      <configuration>
        <createDependencyReducedPom>false</createDependencyReducedPom>
      </configuration>
      <executions>
        <execution>
          <phase>package</phase>
          <goals>
            <goal>shade</goal>
          </goals>
        </execution>
      </executions>
    </plugin>
  </plugins>
</build>

AND I was building the jar file in a wrong way. The best way is to build your jar file is to navigate to the project directory using Terminal or Command Prompt and executing the following command:

mvn package

This includes all the necessary dependencies in your jar file. Your jar file will be located in ./target directory.

Source: https://docs.aws.amazon.com/lambda/latest/dg/java-create-jar-pkg-maven-no-ide.html

Thank you for pointing out the last step! Any idea why Intellij IDEA doesn't build the package correctly?

Charles Letcher · Accepted Answer · 2018-03-18 23:06:09Z

1

I think the issue is that aws-java-sdk-s3 is not provided to lambda functions as part of aws-java-sdk, so you have to include it in your .jar explicitly.

My pom looks like this to clear the error:

        <dependencies>
            <dependency>
                <groupId>com.amazonaws</groupId>
                <artifactId>aws-java-sdk</artifactId>
                <version>1.11.297</version>
                <scope>provided</scope>
            </dependency>
            <dependency>
                <groupId>com.amazonaws</groupId>
                <artifactId>aws-java-sdk-s3</artifactId>
                <version>1.11.297</version>
            </dependency>

answered Mar 18, 2018 at 23:06

Charles Letcher

911 silver badge5 bronze badges

Collectives™ on Stack Overflow

AWS Lambda Java to S3 - java.lang.ClassNotFoundException on com.amazonaws.services.s3.AmazonS3ClientBuilder

2 Answers 2

1 Comment

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Related