AWS Lambda event to python dictonary

Question

I am trying to convert event json from aws lambda function to python dictionary to that I can the event type and cluster identifier but I am getting an error which i am not able to resolve

Below is my code and error

import json
st = """
{
   "Records":[
      {
         "EventSource":"aws:sns",
         "EventVersion":"1.0",
         "EventSubscriptionArn":"arn:aws:sns:us-east-2:427128480143:terraform-redshift-sns-topic:cbdfec04-7502-4509-9954-435e2ddc5e3c",
         "Sns":{
            "Type":"Notification",
            "MessageId":"fecb7b39-a861-5450-9179-23d0ce68f268",
            "TopicArn":"arn:aws:sns:us-east-2:427128480243:terraform-redshift-sns-topic",
            "Subject":"AmazonRedshiftINFO - Cluster Created",
            "Message":"{\"Event Source\":\"cluster\",\"Resource\":\"qa-redshift-cluster\",\"Event Time\":\"2021-04-08 20:12:57.466\",\"Identifier Link\":\"https://console.aws.amazon.com/redshift/home?region=us-east-2#cluster-details:cluster=qa-redshift-cluster \",\"Severity\":\"INFO\",\"Category\":[\"Management\"],\"About this Event\":\"http://docs.aws.amazon.com/redshift/latest/mgmt/working-with-event-notifications.html#REDSHIFT-EVENT-2000 \",\"Event Message\":\"Amazon Redshift cluster \\'qa-redshift-cluster\\' has been created at 2021-04-08 20:12 UTC and is ready for use.\"}",
            "Timestamp":"2021-04-08T20:12:57.905Z",
            "SignatureVersion":"1",
            "Signature":"jS2AK8hf/rebeXMsfFw0DJx+788w+RiDTXyAbLNZzYdE5Mlhi6GRRIns8VaAJZc5otXkkhGshvgvuE0JsUiOhXBccGEJY6Z+lhx6cLSQoEdQB0DRfwltWKOfpQ88LZXJuKCxYcsPL3y5veBdjJqwjdBZtcVYV9BYRhQvT6q/0MpDcJeOYzMkdpR2ULX5B7GasZ3AV0WbKxIjjzFSd3GNud2m85obRrB//NHQwqN6ydvg7PaN/sXuyJjmguRok27O6YNkIqzKjC4JxDbTl4BwyIbck59edbHC5kxmfpoc/RqjF/kUGaqnf0HYOUuMfDT85+9wYVz8vFFER1v3NnKRLA==",
            "SigningCertUrl":"https://sns.us-east-2.amazonaws.com/SimpleNotificationService-010a507c1833636cd94bdd98bd93083a.pem",
            "UnsubscribeUrl":"https://sns.us-east-2.amazonaws.com/?Action=Unsubscribe&SubscriptionArn=arn:aws:sns:us-east-2:427128480243:terraform-redshift-sns-topic:cbdfec04-7502-4509-9954-435e7ddc5e3c",
            "MessageAttributes":{
               
            }
         }
      }
   ]
}
"""

data1 = json.loads(st)
print(type(data1))
print(data1)

data1 = json.loads(st)
print(type(data1))
print(data1)


{'Records': [{'EventSource': 'aws:sns', 'EventVersion': '1.0', 'EventSubscriptionArn': 'arn:aws:sns:us-east-2:427128480243:terraform-redshift-sns-topic:cbdfec04-7502-4509-9954-435e1ddc5e3c', 'Sns': {'Type': 'Notification', 'MessageId': 'fecb7b39-a861-5450-9189-23d0ce68f268', 'TopicArn': 'arn:aws:sns:us-east-2:427128480243:terraform-redshift-sns-topic', 'Subject': '[Amazon Redshift INFO] - Cluster Created', 'Message': '{"Event Source":"cluster","Resource":"qa-redshift-cluster","Event Time":"2021-04-08 20:12:57.466","Identifier Link":"https://console.aws.amazon.com/redshift/home?region=us-east-2#cluster-details:cluster=qa-redshift-cluster ","Severity":"INFO","Category":["Management"],"About this Event":"http://docs.aws.amazon.com/redshift/latest/mgmt/working-with-event-notifications.html#REDSHIFT-EVENT-2000 ","Event Message":"Amazon Redshift cluster \'qa-redshift-cluster\' has been created at 2021-04-08 20:12 UTC and is ready for use."}', 'Timestamp': '2021-04-08T20:12:57.905Z', 'SignatureVersion': '1', 'Signature': 'jS2AK8hf/rebeXMsfFw0DJx+788w+RiDTXyAbLNZzYdE5Mlhi6GRRIns8VaAJZc5otXkkhGshvgvuE0JsUiOhXBccGEJY6Z+lhx6cLSQoEdQB0DRfwltWKOfpQ88LZXJuKCxYcsPL3y5veBdjJqwjdBZtcVYV9BYRhQvT6q/0MpDcJeOYzMkdpR2ULX5B7GasZ3AV0WbKxIjjzFSd3GNud2m85obRrB//NHQwqN6ydvg7SaN/sXuyJjmguRok27O6YNkIqzKjC4JxDbTl4BwyIbck59edbHC5kxmfpoc/RqjF/kUGaqnf0HYOUuMfDT85+9wYVz8vFFER1v3NnKRLA==', 'SigningCertUrl': 'https://sns.us-east-2.amazonaws.com/SimpleNotificationService-010a507c1833636cd94bdb98bd93083a.pem', 'UnsubscribeUrl': 'https://sns.us-east-2.amazonaws.com/?Action=Unsubscribe&SubscriptionArn=arn:aws:sns:us-east-2:427128480243:terraform-redshift-sns-topic:cbdfec04-7502-4509-9954-435e1ddc5e3c', 'MessageAttributes': {}}}]}

Error

Traceback (most recent call last):
  File "/home/deepak/PycharmProjects/TerraformSeriousProject/boto3Examples/src/test.py", line 29, in <module>
    data1 = json.loads(st)
  File "/usr/lib/python3.8/json/__init__.py", line 357, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python3.8/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python3.8/json/decoder.py", line 353, in raw_decode
    obj, end = self.scan_once(s, idx)
json.decoder.JSONDecodeError: Expecting ',' delimiter: line 13 column 26 (char 520)

I used https://jsonformatter.curiousconcept.com/# tool to format the json

apr_1985 · Accepted Answer · 2021-04-09 15:07:02Z

Loading like this in Python you will need to double the backslashes on the Message line. The slash that you have there currently will be used to create the string. You can see this by doing a print of st

However the better way to do this would be to not have st as a string but a dictionary (i.e. remove the triple quotes), then you don't need to mess around with the json library at all here

st = {
    "Records":[
        {
            "EventSource":"aws:sns",
            "EventVersion":"1.0",
            "EventSubscriptionArn":"arn:aws:sns:us-east-2:427128480143:terraform-redshift-sns-topic:cbdfec04-7502-4509-9954-435e2ddc5e3c",
            "Sns":{
                "Type":"Notification",
                "MessageId":"fecb7b39-a861-5450-9179-23d0ce68f268",
                "TopicArn":"arn:aws:sns:us-east-2:427128480243:terraform-redshift-sns-topic",
                "Subject":"AmazonRedshiftINFO - Cluster Created",
                "Message":"{\"Event Source\":\"cluster\",\"Resource\":\"qa-redshift-cluster\",\"Event Time\":\"2021-04-08 20:12:57.466\",\"Identifier Link\":\"https://console.aws.amazon.com/redshift/home?region=us-east-2#cluster-details:cluster=qa-redshift-cluster
    \",\"Severity\":\"INFO\",\"Category\":[\"Management\"],\"About this Event\":\"http://docs.aws.amazon.com/redshift/latest/mgmt/working-with-event-notifications.html#REDSHIFT-EVENT-2000 \",\"Event Message\":\"Amazon Redshift cluster \\'qa-redshift-cluster\\' ha
    s been created at 2021-04-08 20:12 UTC and is ready for use.\"}",
                "Timestamp":"2021-04-08T20:12:57.905Z",
                "SignatureVersion":"1",
                "Signature":"jS2AK8hf/rebeXMsfFw0DJx+788w+RiDTXyAbLNZzYdE5Mlhi6GRRIns8VaAJZc5otXkkhGshvgvuE0JsUiOhXBccGEJY6Z+lhx6cLSQoEdQB0DRfwltWKOfpQ88LZXJuKCxYcsPL3y5veBdjJqwjdBZtcVYV9BYRhQvT6q/0MpDcJeOYzMkdpR2ULX5B7GasZ3AV0WbKxIjjzFSd3GNud2m85obRrB//NHQwqN6ydv
    g7PaN/sXuyJjmguRok27O6YNkIqzKjC4JxDbTl4BwyIbck59edbHC5kxmfpoc/RqjF/kUGaqnf0HYOUuMfDT85+9wYVz8vFFER1v3NnKRLA==",
                "SigningCertUrl":"https://sns.us-east-2.amazonaws.com/SimpleNotificationService-010a507c1833636cd94bdd98bd93083a.pem",
                "UnsubscribeUrl":"https://sns.us-east-2.amazonaws.com/?Action=Unsubscribe&SubscriptionArn=arn:aws:sns:us-east-2:427128480243:terraform-redshift-sns-topic:cbdfec04-7502-4509-9954-435e7ddc5e3c",
                "MessageAttributes":{
                }
            }
        }
]
}
print(st['Records'][0]['Sns']['Message'])
{"Event Source":"cluster","Resource":"qa-redshift-cluster","Event Time":"2021-04-08 20:12:57.466","Identifier Link":"https://console.aws.amazon.com/redshift/home?region=us-east-2#cluster-details:cluster=qa-redshift-cluster ","Severity":"INFO","Category":["Management"],"About this Event":"http://docs.aws.amazon.com/redshift/latest/mgmt/working-with-event-notifications.html#REDSHIFT-EVENT-2000 ","Event Message":"Amazon Redshift cluster \'qa-redshift-cluster\' has been created at 2021-04-08 20:12 UTC and is ready for use."}

Robert Kossendey · Accepted Answer · 2021-04-09 15:07:12Z

You have to escape double quotes in python with two backslashes, like this \\".

This event would work:

st = """
{"Records":[{
         "EventSource":"aws:sns",
         "EventVersion":"1.0",
         "EventSubscriptionArn":"arn:aws:sns:us-east-2:427128480143:terraform-redshift-sns-topic:cbdfec04-7502-4509-9954-435e2ddc5e3c",
         "Sns":{
            "Type":"Notification",
            "MessageId":"fecb7b39-a861-5450-9179-23d0ce68f268",
            "TopicArn":"arn:aws:sns:us-east-2:427128480243:terraform-redshift-sns-topic",
            "Subject":"AmazonRedshift INFO - Cluster Created",
            "Message":"{\\"Event Source\\":\\"cluster\\",\\"Resource\\":\\"qa-redshift-cluster\\",\\"Event Time\\":\\"2021-04-08 20:12:57.466\\",\\"Identifier Link\\":\\"https://console.aws.amazon.com/redshift/home?region=us-east-2#cluster-details:cluster=qa-redshift-cluster \\",\\"Severity\\":\\"INFO\\",\\"Category\\":[\\"Management\\"],\\"About this Event\\":\\"http://docs.aws.amazon.com/redshift/latest/mgmt/working-with-event-notifications.html#REDSHIFT-EVENT-2000 \\",\\"Event Message\\":\\"Amazon Redshift cluster 'qa-redshift-cluster' has been created at 2021-04-08 20:12 UTC and is ready for use.\\"}",
            "Timestamp":"2021-04-08T20:12:57.905Z",
            "SignatureVersion":"1",
            "Signature":"jS2AK8hf/rebeXMsfFw0DJx+788w+RiDTXyAbLNZzYdE5Mlhi6GRRIns8VaAJZc5otXkkhGshvgvuE0JsUiOhXBccGEJY6Z+lhx6cLSQoEdQB0DRfwltWKOfpQ88LZXJuKCxYcsPL3y5veBdjJqwjdBZtcVYV9BYRhQvT6q/0MpDcJeOYzMkdpR2ULX5B7GasZ3AV0WbKxIjjzFSd3GNud2m85obRrB//NHQwqN6ydvg7PaN/sXuyJjmguRok27O6YNkIqzKjC4JxDbTl4BwyIbck59edbHC5kxmfpoc/RqjF/kUGaqnf0HYOUuMfDT85+9wYVz8vFFER1v3NnKRLA==",
            "SigningCertUrl":"https://sns.us-east-2.amazonaws.com/SimpleNotificationService-010a507c1833636cd94bdd98bd93083a.pem",
            "UnsubscribeUrl":"https://sns.us-east-2.amazonaws.com/?Action=Unsubscribe&SubscriptionArn=arn:aws:sns:us-east-2:427128480243:terraform-redshift-sns-topic:cbdfec04-7502-4509-9954-435e7ddc5e3c",
            "MessageAttributes":{

            }
         }
      }
   ]
}
"""

Collectives™ on Stack Overflow

AWS Lambda event to python dictonary

2 Answers 2

1 Comment

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Related