Python logging : Log data to server using the logging module

Question

The logging module offers the possibility of using HTTPHandler, which does not align to my requirement due to the limitations with formatting.

As given in the documentation, https://docs.python.org/3/library/logging.handlers.html, Using setFormatter() to specify a Formatter for a HTTPHandler has no effect.

My aim is to Log events in my application, and collect them on a local server. I am using JSON-Server to mock REST API (https://github.com/typicode/json-server). I have referred to this link : How to set up HTTPHandler for python logging, as a possible solution but i am not able to get what is desired.

My code:

"""class CustomHandler(logging.handlers.HTTPHandler):
    def __init__(self):
        logging.handlers.HTTPHandler.__init__(self)

    def emit(self, record):
        log_entry = self.format(record)
        # some code....
        url = 'http://localhost:3000/posts'
        # some code....
        return requests.post(url, log_entry, json={"Content-type": "application/json"}).content """

def custom_logger(name):

    logger = logging.getLogger(name)

    formatter_json = jsonlogger.JsonFormatter(
        fmt='%(asctime)s %(levelname)s %(name)s %(message)s') 

    requests.post('http://localhost:3000/posts', json= {"message" : "1" } ) 

 
    filehandler_all = logging.FileHandler('test.log')
    filehandler_all.setLevel(logging.DEBUG)
    filehandler_all.setFormatter(formatter_json)           
    logger.addHandler(filehandler_all)


    #http_handler = logging.handlers.HTTPHandler('http://localhost:3000' ,
     #"/posts", "POST")
    #
    # http_handler = CustomHandler()
   # http_handler.setFormatter(formatter_json)  
   # http_handler.setLevel(logging.DEBUG)
   
    return logger

logger = custom_logger("http")
logger.exception("{'sample json message' : '2'}")

The comments are for testing, and ease of reproduction of code.

In the above code snippet, the filehandler handles the json files perfectly, but the HTTPHandler does not. I tried creating a CustomHandler as specified in the link which in principle should work but I am not able to figure it out the details.

Will constructing a CustomHandler with changes to the "mapLogRecord" and the "emit" methods make sense?.

The most important thing, is to get the data in JSON Format.

Any other ideas to approach this can also be helpful!

changes to mapLogRecord make sense. changes to emit do not make sense. the fundamental issue is that what is logged are not strings, but LogRecord objects, and these objects need to be turned into strings some way. — blues
– blues, Commented Jul 17, 2020 at 17:46
Thanks, that made sense to proceed in the right direction. Another thing, is to maybe use the HTTP Handler directly, and filter out the log messages on server. This is my last approach, will update this question as and when I find a solution. — DragonsCanDance
– DragonsCanDance, Commented Jul 20, 2020 at 7:35

DragonsCanDance · Accepted Answer · 2020-07-21 12:12:42Z

Okay, so this is a solution which can output logs in JSON Format on server. It uses a custom logger. I am also able to format the messages for suitability. The code below gives the output in json format and uses the request module.

class RequestsHandler(logging.Handler):
    def emit(self, record):
        log_entry = self.format(record)
        return requests.post('http://localhost:3000/posts',
                             log_entry, headers={"Content-type": "application/json"}).content

class FormatterLogger(logging.Formatter):
    def __init__(self, task_name=None):
        
        super(FormatterLogger, self).__init__()

    def format(self, record):
        data = {'@message': record.msg,                
                 '@funcName' : record.funcName,
                 '@lineno' : record.lineno,
                 '@exc_info' : record.exc_info, 
                 '@exc_text' : record.exc_text,                 
                 }           

        return json.dumps(data)



def custom_logger(name):
    logger = logging.getLogger(name)
    custom_handler = RequestsHandler()
    formatter = FormatterLogger(logger)
    custom_handler.setFormatter(formatter)
    logger.addHandler(custom_handler)
    
    return logger


logger = custom_logger("http")
logger.exception("{'sample json message' : '2'}")

The data variable controls the parameters that can be added to the message.

The output :

 {
      "@message": "{'sample json message' : '2'}",
      "@funcName": "<module>",
      "@lineno": 62,
      "@exc_info": [
        null,
        null,
        null
      ],
      "@exc_text": null,
      "id": 138
    }

Collectives™ on Stack Overflow

Python logging : Log data to server using the logging module

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related