Aws Lambda response error

Question

I am running aws lambda which will fetch data from maria DB and return the fetched rows as a JSON object. A total number of item in JSON array is 64K.

I am getting this error:

{ "error": "body size is too long" }

Is there a way I can send all 64K rows by making any configuration change to lambda?

Ashan · Accepted Answer · 2017-09-19 11:10:07Z

27

You cannot send the 64K rows (Which goes beyond 6MB body payload size limit) making configuration changes to Lambda. Few alternative options are.

Query the data and build a JSON file with all the rows in /tmp (Up to 512MB) directory inside Lambda, upload it to S3 and return a CloudFront Signed URL to access the data.
Split the dataset into multiple pages and do multiple queries.
Use a EC2 instance or ECS, instead of Lambda.

Note: Based on the purpose of queried data, its size & etc. different mechanisms can be used, efficiently using other AWS services.

answered Sep 19, 2017 at 11:10

Ashan

19.7k5 gold badges53 silver badges69 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Irfan Pathan Over a year ago

Dear can you explain me your third point in more details..Do you mean writing API in node js with express and publishing it on EC2 instance ?

Ashan Over a year ago

Yes. Either using NodeJS (With ExpressJS or any other suitable framework) in a EC2 instance or using Docker containers, host the service in Container Cluster.

Irfan Pathan Over a year ago

Thanks a lot @Ashan for the suggestion, I am working on it now...Cheers

Vijayanath Viswanathan · Accepted Answer · 2017-09-19 10:38:18Z

3

This error indicates that your response exceeds the maximum (6 MB), which is maximum data size AWS Lambda can respond.

http://docs.aws.amazon.com/lambda/latest/dg/limits.html

answered Sep 19, 2017 at 10:38

Vijayanath Viswanathan

8,6713 gold badges31 silver badges50 bronze badges

Comments

gjtempleton · Accepted Answer · 2017-09-19 10:38:07Z

1

It seems that you're hitting the hard limit of a maximum 6 MB response size. As it's a hard limit there's unfortunately no way to increase this.

You'll need to set up your lambda to be able to send at most 6MB and paginate through the rows you need to retrieve in different invocations until you've fetched all 64K.

Sources: https://docs.aws.amazon.com/lambda/latest/dg/limits.html#limits-list https://forums.aws.amazon.com/thread.jspa?threadID=230229

answered Sep 19, 2017 at 10:38

gjtempleton

8038 silver badges16 bronze badges

Collectives™ on Stack Overflow

Aws Lambda response error

3 Answers 3

3 Comments

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

3 Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related