AWS Lambda is not invoked if byte array is published via mqtt

Question

I have created a rule in AWS-iot to invoke a lambda with following query:

select * from 'my_topic'

This rule will invoke a below javascript lambda:

var AWS = require('aws-sdk');
var zlib = require('zlib');
var s3 = new AWS.S3();

        exports.handler = (event, context, callback) => {  
            console.log("hello:");
            var bucketName = "otonomobucket";
            var keyName = getKeyName("myData", Date.now());
            var content = event;
            //var content = JSON.stringify(event);

            console.log("event data:" + content);

            var params = { Bucket: bucketName, Key: keyName, Body: content };

            s3.putObject(params, function (err, data) {
                if (err)
                    console.log(err)
                else
                    console.log("Successfully saved object to " + bucketName + "/" + keyName);
            });
        };

        function getKeyName(folder, filename) {
            return folder + '/' + filename + ".txt";
        }

The lambda works perfectly if i publish any string to my_topic. But the lambda is not working if i publish bytearray to my_topic. CloudWatch is also not showing any logs. How could i get the byte array into my lambda function.

Ben T · Accepted Answer · 2020-02-20 00:31:25Z

4

The lambda can receive binary data as long as it is base64 encoded and in a JSON payload. This can be done by changing the rule to:

SELECT encode(*, 'base64') AS data FROM 'my_topic'

The documentation on binary payloads in the SELECT clause indicates that SELECT * FROM 'a/b' should work. But in my experience this only works for non-JSON payloads with some actions (e.g. writing to S3) and not with others (sending it to a lambda or Kinesis stream).

This appears to be an issue with how the lambda is invoked.

From https://forums.aws.amazon.com/thread.jspa?messageID=925603&#925603

AWS Lambda only accepts JSON payloads [1] so the Lambda action will only work for json payloads. This is a common problem and we recommend modifying your rule to base64 encode the binary into a json blob like so:

select encode(*, 'base64') as data from 'some/topic'

Your Lambda function will then need to be modified to decode the 'data' key in the event body.

[1] https://docs.aws.amazon.com/lambda/latest/dg/API_Invoke.html#API_Invoke_RequestBody

My hypothesis is that the lambda invocation uses the lambda REST API. This requires a JSON payload in the request body as described in the link to the API_Invoke_RequestBody

edited Feb 20, 2020 at 0:31

answered Feb 19, 2020 at 12:12

Ben T

4,9963 gold badges25 silver badges24 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

nnn Over a year ago

It is a well explained answer to the question. Thanks, you saved my day. :)

Collectives™ on Stack Overflow

AWS Lambda is not invoked if byte array is published via mqtt

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related