AWS Lambda call url which takes longer than Lambda execution limit

Question

Context

I want to run an AWS Lambda, call an endpoint (fire and forget) then stop the Lambda - all the while the endpoint is whirling away happily on it's own.

Attempts

1.

Using a timeout, e.g.

try:
    requests.get(url, timeout=0.001)

except requests.exceptions.ReadTimeout:
    ...

2.

Using an async call with grequest:

import grequests
...

def handler(event, context):

    try:
        ...

        req = grequests.get(url)
        grequests.send(req, grequests.Pool(1))

        return {
            'statusCode': 200,
            'body': "Lambda done"
        }

    except Exception:
        logger.exception('Error while running lambda')
        raise

These requests don't appear to reach the API, it's almost like the request is being cancelled.

Any ideas why?

Question

How can a Lambda call a URL which takes a long time to complete? Thanks.

How long is long time? AWS will kill your lambda function after 300 seconds. See: docs.aws.amazon.com/lambda/latest/dg/limits.html — mehdix
– mehdix, Commented Sep 20, 2018 at 11:33
the url you are trying to hit from lambda, is it a public endpoint? Is you lambda inside the vpc? if so it should be launched inside a public subnet, check your n/w configuration. — Vaisakh PS
– Vaisakh PS, Commented Sep 20, 2018 at 13:23
@VaisakhPS a lambda function in a public subnet will not have Internet access, since lambda functions don't get public IP addresses assigned to them regardless of subnet they are in. If the Lambda is in a VPC, then it must be in a private subnet with a NAT gatetway in order to have Internet access. — Mark B
– Mark B, Commented Sep 20, 2018 at 13:32
@greenlamponatable Are you sure your function has network access to connect to the API endpoint? Is the API endpoint public or inside your VPC? Are you deploying the Lambda function inside a VPC? — Mark B
– Mark B, Commented Sep 20, 2018 at 13:33

greenlamponatable · Accepted Answer · 2018-09-21 17:12:44Z

1

To anyone reading this: I fixed my problem by using AWS Batch.

answered Sep 21, 2018 at 17:12

greenlamponatable

1383 silver badges13 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

mehdix · Accepted Answer · 2018-09-20 11:39:47Z

0

You have to increase the timeout on your function. Depending on how you have defined your function, you can increase the default timeout of 300 seconds on the function page or inside your cloudformation script.

The timeout defined inside your function has no effect on lambda functionality. AWS Lambda will kill your function after 300 seconds regardless of any timeout defined inside your script. See: https://docs.aws.amazon.com/lambda/latest/dg/limits.html

answered Sep 20, 2018 at 11:39

mehdix

5,2042 gold badges31 silver badges38 bronze badges

Collectives™ on Stack Overflow

AWS Lambda call url which takes longer than Lambda execution limit

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related