Python Logging - Add addifinal fields to Custom Formatter

Question

I have got the following formatter class for a Logger, which is used in our companies libary:

import json
import logging
from typing import Optional


class CustomFormatter(logging.Formatter):

    def __init__(
        self,
        fmt: Optional[str] = "%(asctime)s",
        datefmt: Optional[str] = None,
        style: Optional[str] = "%",
        confidentiality: Optional[str] = "C3",
    ) -> None:
        self.confidentiality = confidentiality
        super().__init__(fmt=fmt, datefmt=datefmt, style=style)

    def formatMessage(self, record: logging.LogRecord, *args, **kwargs) -> str:
        super().formatMessage(record)
        return json.dumps(
            {
                "asctime": record.asctime,
                "level": record.levelname,
                "name": record.name,
                "message": record.message,
                "timeMillis": int(record.created * 1000),
                "pathName": record.pathname,
                "funcName": record.funcName,
                "lineNumber": record.lineno,
                "confidentiality": self.confidentiality,
            }
        )

I tried to add custom fields like this:

old_factory = logging.getLogRecordFactory()

def record_factory(*args, **kwargs):
    record = old_factory(*args, **kwargs)
    record.custom_attribute = "my-attr"
    return record

logging.setLogRecordFactory(record_factory)

logger = logging.getLogger()
logger.setLevel(logging.DEBUG)
logger_handler = logging.StreamHandler()
logger_handler.setFormatter(CustomFormatter())
logger.addHandler(logger_handler)

logger.info("Ich bin ein Test")

Result:

{"asctime": "2022-06-29 10:40:22,869", "level": "INFO", "name": "root", "message": "test", "timeMillis": 1656492022869, "pathName": "C:\\Users\\xcg5847\\Desktop\\loggingnew\\test.py", "funcName": "<module>", "lineNumber": 15, "
confidentiality": "C3"}

This is not being added. I guess the problem is that, the formatter always returns that same json object and that this is per se not extensible the way it was build.

This is what I desire:

{"asctime": "2022-06-29 10:40:22,869", "level": "INFO", "name": "root", "message": "test", "timeMillis": 1656492022869, "pathName": "C:\\Users\\xcg5847\\Desktop\\loggingnew\\test.py", "funcName": "<module>", "lineNumber": 15, "
confidentiality": "C3", "custom_attribute": "my-attr"}

IMPORTANT: Ideally this would work like this:

logger.info("test", extra={"custom_attribute": "my-attr"}

blues · Accepted Answer · 2022-06-29 09:12:06Z

2

The Problem you are facing is, that you successfully add the attribute to the Record but your formatter just ignores it. There is no way around that but to either change or replace the CustomFormatter that you are using. A possible solution could look like this:

class CustomFormatterExtra(CustomFormatter):
    custom_name = 'custom_attribute'

    def formatMessage(self, record, *args, **kwargs):
        json_str = super().formatMessage(record, *args, **kwargs)
        if hasattr(record, self.custom_name):
            json_dict = json.loads(json_str)
            json_dict[self.custom_name] = getattr(record, self.custom_name)
            json_str = json.dumps(json_dict)
        return json_str


logger = logging.getLogger()
logger.setLevel(logging.DEBUG)
logger_handler = logging.StreamHandler()
logger_handler.setFormatter(CustomFormatterExtra())
logger.addHandler(logger_handler)

logger.info("Ich bin ein Test", extra={'custom_attribute':123})
CustomFormatterExtra.custom_name = 'different_attribute'
logger.info("Ich bin ein Test", extra={'different_attribute':123})

Note that this does not need a custom LogRecord factory. The extra param does exactly what yours did.

answered Jun 29, 2022 at 9:12

blues

5,2743 gold badges26 silver badges46 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Data Mastery Over a year ago

thanks, updated it a bit and works totally fine :-)

Collectives™ on Stack Overflow

Python Logging - Add addifinal fields to Custom Formatter

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related