AWS Lambda function using JavaScript SDK always throws timeout error

Question

I am attempting to use the AWS SDK for JavaScript within my AWS Lambda function (NodeJS 6.10 runtime). My ultimate goal is to use this to manage my ECS instances, but for now I'm simply trying to use any part of the API and am failing with each attempt. I have reduced the function to the simplest possible; take a look:

exports.handler = (event, context, callback) => {
    var AWS = require('aws-sdk');
    (new AWS.ECS({"apiVersion": '2014-11-13'})).listClusters({}, (err, data) => {
        if (err) console.log(err, err.stack);
        else     console.log(data);

        callback(null, "DONE");
    })
};

I have given this function an IAM role that has this definition:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "aws:*",
                "ecs:*"
            ],
            "Resource": [
                "*"
            ]
        }
    ]
}

I have set this function to run within my existing VPC, subnets and security groups. I have increased the timeouts and memory caps beyond all possible needs.

Every execution of this function fails with a timeout exception. I have tried using many different API calls, and even different service APIs, but every attempt to invoke an API within my function always results in a timeout.

I even enabled X-Ray tracing for this function, but by all appearances it seems that nothing leaves the Lambda execution environment - X-Ray reports no activity out to other parts of AWS (ECS, for example).

What have I missed? Why can't I use any of the JS SDK within Lambda?

I don't think an action of "aws:*" does anything, because it is not the name of a service. — John Rotenstein
– John Rotenstein, Commented Apr 30, 2017 at 21:28
@JohnRotenstein yep aws:* was just one of many failed attempts. — Jake Feasel
– Jake Feasel, Commented May 1, 2017 at 4:07

Community · Accepted Answer · 2017-05-23 12:17:56Z

3

Your code worked perfectly fine for me, but here's some things I had to do first:

Used the VPC Wizard to create a VPC with a Public and Private Subnets and a NAT instance
Configured the Lambda function to use the Private Subnet
Gave the Lambda function additional permissions to CreateNetworkInterface (See Lambda creating ENI everytime it is invoked: Hitting limit)

The above is because the Lambda function requires Internet access to cal the AWS API endpoints. Lambda functions attached to a VPC only have a Private IP address, so they require a NAT Gateway or NAT Instance to have Internet access. See: Internet Access for Lambda Functions

edited May 23, 2017 at 12:17

CommunityBot

11 silver badge

answered Apr 30, 2017 at 21:27

John Rotenstein

273k28 gold badges456 silver badges541 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

Jake Feasel Over a year ago

You are correct - the key seems to be the VPC configuration. When I disassociate the Lambda function from the VPC it works fine. However, ultimately I need my function to access the resources within my VPC. So I guess this is the crux of the issue - how to configure my VPC properly so the SDK works from within Lambda? I must be doing something wrong here. I'll continue investigating. Thanks

John Rotenstein Over a year ago

To call the SDK from within Lambda, put it in a private subnet with a NAT Gateway or a NAT Instance.

Jake Feasel Over a year ago

I saw the suggestion to do that very thing after submitting my question, and spent quite a long time attempting to do so. No luck so far.

Jake Feasel Over a year ago

John, I have been successful (finally!) with getting all of this sorted out. I have posted an answer which provided the details that I struggled with, hopefully all of this will help others dealing with this problem. Thanks again.

Jake Feasel · Accepted Answer · 2017-05-29 16:29:18Z

For posterity, I took John Rotenstein's excellent advise and used the VPC Wizard to get things working. Turns out the pivotal detail has to do with the construction of the NAT gateway.

If you want your Lambda function to have access to the AWS SDK and your VPC resources, you need at least two subnets (one for public and one for private resources). Here are the VPC commands you need to execute, which are similar to what happens with the VPC Wizard:

create a new VPC, or set these env variables to your existing ones

export REGION=us-west-2 #or whatever region you want

export VPC_ID=`aws ec2 create-vpc --cidr-block 10.1.0.0/16 \
    --query Vpc.VpcId --output text`

create an internet gateway for your public resources

export IGW_ID=`aws ec2 create-internet-gateway \
    --query InternetGateway.InternetGatewayId --output text`

aws ec2 attach-internet-gateway --internet-gateway-id $IGW_ID --vpc-id $VPC_ID

create route tables for your public subnet, using the above IGW

export ROUTE_TABLE_ID_PUBLIC=`aws ec2 describe-route-tables \
    --filter Name=vpc-id,Values=$VPC_ID --query RouteTables[0].RouteTableId --output text`

export SUBNET_ID_PUBLIC=`aws ec2 create-subnet --vpc-id $VPC_ID \
    --cidr-block 10.1.0.0/24 --availability-zone ${REGION}a \
    --query Subnet.SubnetId --output text`

aws ec2 associate-route-table --subnet-id $SUBNET_ID_PUBLIC \
    --route-table-id $ROUTE_TABLE_ID_PUBLIC

aws ec2 create-route --route-table-id $ROUTE_TABLE_ID_PUBLIC \
    --gateway-id $IGW_ID --destination-cidr-block 0.0.0.0/0

create an IP Allocation for a NAT Gateway, for use with a private subnet

export IP_ALLOCATION_ID=`aws ec2 allocate-address --domain vpc \
    --query AllocationId --output text`

export ROUTE_TABLE_ID_PRIVATE=`aws ec2 create-route-table --vpc-id $VPC_ID \
    --query RouteTable.RouteTableId --output text`

export SUBNET_ID_PRIVATE=`aws ec2 create-subnet --vpc-id $VPC_ID \
    --cidr-block 10.1.1.0/24 --availability-zone ${REGION}b \
    --query Subnet.SubnetId --output text`

aws ec2 associate-route-table --subnet-id $SUBNET_ID_PRIVATE \
    --route-table-id $ROUTE_TABLE_ID_PRIVATE

export NAT_GW_ID=`aws ec2 create-nat-gateway --subnet-id $SUBNET_ID_PUBLIC \
    --allocation-id $IP_ALLOCATION_ID --query NatGateway.NatGatewayId --output text`

Wait here a few moments - it takes some time before the NAT Gateway is ready and usable for further commands.

The key detail (that I was unable to find in the AWS documentation) is within that last command, above - that the NAT Gateway must be created with the PUBLIC subnet, even though it is associated with the PRIVATE route table:

create private route table with NAT Gateway

aws ec2 create-route --route-table-id $ROUTE_TABLE_ID_PRIVATE \
    --gateway-id $NAT_GW_ID --destination-cidr-block 0.0.0.0/0

create security groups, etc....

export SECURITY_GROUP_ID=`aws ec2 create-security-group --vpc-id $VPC_ID \
    --group-name mygroup --description "My SG" \
    --query GroupId --output text`

aws ec2 authorize-security-group-ingress --group-id $SECURITY_GROUP_ID \
    --protocol tcp --port 22 --cidr 0.0.0.0/0

aws ec2 authorize-security-group-ingress --group-id $SECURITY_GROUP_ID \
    --cidr 10.1.0.0/16 --protocol all

At this point you should be able to create Lambda functions which are associated with the private subnet, which have access to resources in the VPC and also can make calls out to the Internet (necessary for AWS SDK usage). Here's an example Lambda function which does exactly this:

create IAM roles necessary to execute Lambda functions within your VPC

aws iam create-instance-profile --instance-profile-name testRole

testRole_trust_policy.json:
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": "lambda.amazonaws.com"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}

export TEST_ROLE_ARN=`aws iam create-role --role-name testRole \
  --assume-role-policy-document file://testRole_trust_policy.json \
  --query Role.Arn --output text`


testRole_policy.json:
{
  "Version": "2012-10-17",
  "Statement": [
      {
          "Effect": "Allow",
          "Action": [
              "lambda:*",
              "ec2:CreateNetworkInterface",
              "ec2:DescribeNetworkInterfaces",
              "ec2:DeleteNetworkInterface"
          ],
          "Resource": [
              "*"
          ]
      }
  ]
}

aws iam put-role-policy --role-name testRole \
  --policy-name testRole --policy-document file://testRole_policy.json

aws iam add-role-to-instance-profile \
  --instance-profile-name testRole \
  --role-name testRole

create a Lambda function using this IAM role

contents of lambda.zip:
- test.js:
    exports.handler = (event, context, callback) => {
        AWS = require('aws-sdk'),
            lambda = new AWS.Lambda({"apiVersion": '2015-03-31'});

        lambda.listFunctions(callback);
    };


aws lambda create-function --function-name testWithVPC \
    --runtime nodejs6.10 --role $TEST_ROLE_ARN \
    --handler test.handler --timeout 10 \
    --zip-file fileb://lambda.zip \
    --vpc-config SubnetIds=$SUBNET_ID_PRIVATE,SecurityGroupIds=$SECURITY_GROUP_ID

execute it and see the results:

aws lambda invoke --function-name testWithVPC with.txt
with.txt:
    {"NextMarker":null,"Functions":[{"FunctionName":"testWithVPC","FunctionArn": ....]}

This is enough to demonstrate the functionality. My project building upon this pattern is available here, for more robust samples: https://github.com/jakefeasel/sqlfiddle3

Collectives™ on Stack Overflow

AWS Lambda function using JavaScript SDK always throws timeout error

2 Answers 2

4 Comments

create a new VPC, or set these env variables to your existing ones

create an internet gateway for your public resources

create route tables for your public subnet, using the above IGW

create an IP Allocation for a NAT Gateway, for use with a private subnet

create private route table with NAT Gateway

create security groups, etc....

create IAM roles necessary to execute Lambda functions within your VPC

create a Lambda function using this IAM role

execute it and see the results:

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

4 Comments

create a new VPC, or set these env variables to your existing ones

create an internet gateway for your public resources

create route tables for your public subnet, using the above IGW

create an IP Allocation for a NAT Gateway, for use with a private subnet

create private route table with NAT Gateway

create security groups, etc....

create IAM roles necessary to execute Lambda functions within your VPC

create a Lambda function using this IAM role

execute it and see the results:

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related