python logging in aws lambda

Question

Something just doesn't click internally for me with pythons logging despite reading the documentation.

I have this code

import logging
logging.basicConfig(level=logging.INFO,format='%(levelname)s::%(message)s')
LOG = logging.getLogger("__name__")
LOG.info("hey")

If I run it from bash I get this:

INFO::hey

If I run it in an aws lambda the "hey" doesn't shows up at all in the logs.

I then did a test setting the level on the logger by adding this:

LOG.setLevel(logging.INFO)

Run from bash I get the same thing I got before (desired format), but run from the lambda this shows up in the logs:

[INFO] 2022-02-14T23:30:43.39Z eb94600a-af45-4124-99b6-d9651d6a3cf6 hey

Okay... that is pretty odd. The format is not the same as from bash.

I thought I could rationalize the first example because the output on bash is actually going to stderr. And I then assumed the aws lamdba logs just don't grab that. But the second example is also going to stderr on bash, yet it shows up in the lambda logs but with the wrong format. So clearly I am missing something.

What is going on under the hood here?

FYI, in AWS Lambda you can simply use print() and the output will appear in the log in CloudWatch Logs. Just mentioning in case that might satisfy your needs. — John Rotenstein
– John Rotenstein, Commented Feb 15, 2022 at 0:48
@JohnRotenstein yes thanks, I did ran into that. The original script was run on an instance and logging to a file. But at this point I just can't stand that I am not fully getting this logger. I tried several other things that got results I couldn't explain as well. — Jack-of-some
– Jack-of-some, Commented Feb 15, 2022 at 1:21

Anon Coward · Accepted Answer · 2022-02-15 03:36:22Z

7

When your Lambda runs, a harness is running that does some basic bootstrap and then loads your module and invokes it. Part of that bootstrap in the AWS Lambda Python Runtime replaces the standard Python logger with its own:

    logger_handler = LambdaLoggerHandler(log_sink)
    logger_handler.setFormatter(
        logging.Formatter(
            "[%(levelname)s]\t%(asctime)s.%(msecs)dZ\t%(aws_request_id)s\t%(message)s\n",
            "%Y-%m-%dT%H:%M:%S",
        )
    )
    logger_handler.addFilter(LambdaLoggerFilter())

This behavior is formally documented by AWS as AWS Lambda function logging in Python , under the section "Logging library".

answered Feb 15, 2022 at 3:36

Anon Coward

11k3 gold badges31 silver badges42 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Jack-of-some Over a year ago

Now that is interesting. I saw the last link. And it didn't give me any impression it was doing all that. But now that you mention it, the default format was different and I just missed that clue. Thanks.

Collectives™ on Stack Overflow

python logging in aws lambda

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related