11

I am trying to use encrypted environment variables in an AWS Lambda function running in Node.js 4.3, but the code hangs when trying to decrypt the variables. I don't get any error messages, it just times out. Here is what I have tried:

I created the encryption key in the same region as the Lambda, and ensured that the role the Lambda runs as has access to the key. (I've even tried giving the role full control of the key.)

When creating the Lambda, I enable encryption helpers, select my encryption key, and encrypt the environment variable:

enter image description here

Next I click the "Code" button which gives me javascript code that's supposed to handle the decryption at runtime. Here is the code--the only change I have made is to add console.log statements and I added a try/catch:

"use strict";

const AWS = require('aws-sdk');

const encrypted = process.env['DBPASS'];
let decrypted;


function processEvent(event, context, callback) {
    console.log("Decrypted: " + decrypted);
    callback();
}

exports.handler = (event, context, callback) => {
    if (decrypted) {
        console.log('data is already decrypted');
        processEvent(event, context, callback);
    } else {
        console.log('data is NOT already decrypted: ' + encrypted);
        // Decrypt code should run once and variables stored outside of the function
        // handler so that these are decrypted once per container
        const kms = new AWS.KMS();
        console.log('got kms object');
        try {
        var myblob = new Buffer(encrypted, 'base64');
        console.log('got blob');
        kms.decrypt({ CiphertextBlob: myblob }, (err, data) => {
            console.log('inside decrypt callback');
            if (err) {
                console.log('Decrypt error:', err);
                return callback(err);
            }
            console.log('try to get plaintext');
            decrypted = data.Plaintext.toString('ascii');
            console.log('decrypted: ' + decrypted);
            processEvent(event, context, callback);
        });
        }
        catch(e) {
            console.log("exception: " + e);
            callback('error!');
        }
    }
};

Here is what I get when I run the function:

data is NOT already decrypted: AQECAH.....
got kms object
got blob
END RequestId: 9b7af.....
Task timed out after 30.00 seconds

When I run the function, it times out. I see that it prints all log statements up to "got blob" then it just stops. No error message other than timed out. I've tried increasing timeout and memory for the Lambda but it just makes it wait longer before timing out.

How is decryption supposed to work when I never tell the app what decryption key to use? The documentation for decrypt does not mention any way to tell it what decryption key to use. And I am not getting any error messages that would tell me it doesn't know what key to use or anything.

I've tried going through this tutorial but it just tells me to do the same thing I've already done. I've also read all of the environment variables documentation but it says that what I'm doing should just work.

5
  • Is your Lambda function running inside a VPC? Commented Feb 8, 2017 at 17:29
  • @garnaat yes it is in a VPC. It is part of a security group and subnet that is able to connect to an RDS MySQL instance Commented Feb 8, 2017 at 17:35
  • 2
    I think the issue is that you need internet access to make the call to KMS since it is hitting the KMS endpoints. There is no VPC endpoint for KMS so you would need to set up a NAT server in your VPC. Commented Feb 8, 2017 at 17:38
  • @garnaat yes that was the issue. if you can convert your comment to an answer i can accept it. Commented Feb 9, 2017 at 17:29
  • Side note: Depending on how often your Lambda will be invoked, to reduce cost of your KMS usage, you may want to consider storing a symmetric key in KMS and decrypt that outside of your handler. Then use it inside the handler to decrypt the real objects. Commented Jun 10, 2017 at 21:48

1 Answer 1

13

Decrypting the environment variables requires an API call to the KMS service. To do that, your Lambda function must have access to the internet since there are no VPC endpoints for KMS. So, if your Lambda is running in a VPC, make sure you have a NAT configured for the VPC to allow your Lambda function to call KMS.

Sign up to request clarification or add additional context in comments.

1 Comment

There is now the option to add a KMS endpoint to your VPC docs.aws.amazon.com/kms/latest/developerguide/…

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.