Using an EventBridge event pattern string in a lambda function

Question

I have a lambda function using Python. It's connected to an EventBridge rule that triggers every time there's a change in a Glue table.

The event pattern it's outputting looks something like this:

{
    "version":"0",
    "detail":{
        "databaseName":"flights-db",
        "typeOfChange":"UpdateTable",
        "tableName":"flightscsv"
    }
}

I want to get the tableName and databaseName values from this output into the function as a variable.

My Lambda function:

import json
import boto3

def lambda_handler(event, context):
    boto3_version = boto3.__version__

    return_statement = 'Boto3 version: ', boto3_version,\
                       'Event output: ', event

    return {
        'statusCode': 200,
        'body': json.dumps(return_statement)
    }

I was expecting to get the event pattern output from the event in my return statement but that's not the case.

When testing this function the return output for event is:

{\"key1\": \"value1\", \"key2\": \"value2\", \"key3\": \"value3\"}

This key and values are defined like this in the Test Pattern for the function.

The eventbridge rule is defined like this:

How can I get the values from the event pattern to a variable? Do I need to configure the test pattern to get the results into event?

EDIT: Picture of log events for the table change event:

What is the outcome of print(event) when the function is triggered. Also can you show your EB rule that triggers the lambda? — Marcin
– Marcin, Commented Feb 9, 2021 at 8:19
Thanks, but I mean the print(event) when your function is actually invoked by EB, not when you test it in console. The event objects will be different in both cases. — Marcin
– Marcin, Commented Feb 9, 2021 at 8:55
I see. The rule is active and I'm triggering it with a table change but don't know where I can see this print(event) information. Do you know where I can check it? Maybe CloudWatch? — ire
– ire, Commented Feb 9, 2021 at 9:10

Marcin · Accepted Answer · 2021-02-09 09:50:06Z

7

+50

The event object generated by CloudWatch (CW) Events / Event Bridge (EB) are listed here. These events will be passed to your function when it is going to get triggered by EB.

Your EB Event Pattern should be:

{
  "source": ["aws.glue"],
  "detail-type": ["Glue Data Catalog Table State Change"]
}

The above should match changes to any tables in your glue catalog. The event should be similar to the one below:

{
    "version": "0",
    "id": "2617428d-715f-edef-70b8-d210da0317a0",
    "detail-type": "Glue Data Catalog Table State Change",
    "source": "aws.glue",
    "account": "123456789012",
    "time": "2019-01-16T18:16:01Z",
    "region": "eu-west-1",
    "resources": [
        "arn:aws:glue:eu-west-1:123456789012:table/d1/t1"
    ],
    "detail": {
        "databaseName": "d1",
        "changedPartitions": [
            "[C.pdf, dir3]",
            "[D.doc, dir4]"
        ],
        "typeOfChange": "BatchCreatePartition",
        "tableName": "t1"
    }
}

Thus, to get tableName and databaseName your lambda function could be:

import json
import boto3

def lambda_handler(event, context):
    boto3_version = boto3.__version__

    print(event)

    table_name = event['detail']['tableName']
    database_name = event['detail']['databaseName']

    print(table_name, database_name)

    return_statement = {
        'boto3_version': boto3_version,
        'table_name': table_name,
        'database_name': database_name
    }

    return {
        'statusCode': 200,
        'body': json.dumps(return_statement)
    }

For testing, you can setup sample EB event in your lambda test window:

edited Feb 9, 2021 at 9:50

answered Feb 9, 2021 at 9:27

Marcin

241k16 gold badges315 silver badges368 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

ire Over a year ago

Thank you. I'm still having trouble locating where in CloudWatch I can see the print(event). I located my event (table change) under Cloudwatch > Log Groups but I can't see the information. I'll update the post with the picture of log events for the table change event.

Marcin Over a year ago

@ire CloudWatch Logs. There should be a log group for your function. If its not there, its either not triggered or your lambda execution role is missing CW permissions. Can you check that?

ire Over a year ago

I attached the picture of log events in the main post. It's very possible that it's an issue with permissions. Will look into that now and let you know.

Marcin Over a year ago

@ire I also updated the answer with how to use sample EB event for Glue as test in your lambda console. Its easier to test your function with this event.

ire Over a year ago

Got it working on my usecase :) I'll give the reward once it lets me

Collectives™ on Stack Overflow

Using an EventBridge event pattern string in a lambda function

1 Answer 1

5 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

5 Comments

Your Answer

Sign up or log in

Post as a guest

Related