22

If one iswriting a Django/ Google App Engine application and would like to have logs that are conveniently conspicuous based on color (i.e. errors in red), how does one set that up?

I've copied the helpful solution from this question, but I'm not sure how to integrate it into Django/Google App Engine.

I figured one would put the following in the application's main.py (i.e. essentially from the example here: Running Django on Google App Engine):

from contrib.utils import ColouredLogger # from the SO question above
logging.setLoggerClass(ColouredLogger)

... where contrib.utils is where I put airmind's code from the above link to his SO answer.

However, that doesn't seem to do anything to the output to the console for GAE, which continues to be in the original format + plain color.

12 Answers 12

24

We use colorlog and it does exactly what you expect.

For posterity, the formatter config we use is:

'color': {
    '()': 'colorlog.ColoredFormatter',
    'format': '%(log_color)s%(levelname)-8s %(message)s',
    'log_colors': {
        'DEBUG':    'bold_black',
        'INFO':     'white',
        'WARNING':  'yellow',
        'ERROR':    'red',
        'CRITICAL': 'bold_red',
    },
}
Sign up to request clarification or add additional context in comments.

2 Comments

Note that colorlog module does not support 256 colours.
A link, to official docs about logging will be helpful as well. docs.djangoproject.com/en/dev/topics/logging/#examples
13

Although this is not quite what the OP wanted (each line coloured by level), I wanted to share a nice alternative for log output colouring called rich - a fantastic library for various rich text stuff to display in the terminal, authored by @will-mcgugan.

Simple example

Activating rich colouring for Django logs is easy: use rich.logging.RichHandler instead of logging.StreamHandler, e.g.

# project/settings.py

...

LOGGING = {
    'version': 1,
    'disable_existing_loggers': False,
    'handlers': {
        'console': {
            'class': 'rich.logging.RichHandler',  # <-- this
        },
    },
    'root': {
        'handlers': ['console'],
        'level': 'INFO',
    },
    'loggers': {
        'django': {
            'handlers': ['console'],
            'level': os.getenv('DJANGO_LOG_LEVEL', 'INFO'),
            'propagate': False,
        },
    },
}

(this is the modified example from Django logging docs). This will produce terminal logs like these:

django-rich-logs

Changing output format

Format customizations are done as usual by passing the handler arguments. E.g. to turn on rich tracebacks and hide the timestamp column:

LOGGING = {
    ...
    'handlers': {
        'console': {
            'class': 'rich.logging.RichHandler',
            'rich_tracebacks': True,
            'show_time': False,
        },
    },
}

will yield

enter image description here

Changing output colours

Changing the colouring is possible, but not via logging config, as multiple colours and styles are applied to each line; you must provide a customized theme. An example that changes colouring of INFO label from the default blue to bold magenta:

import rich
import rich.theme

my_theme = rich.theme.Theme({
    'logging.level.info': 'bold magenta',
})
rich.reconfigure(theme=my_theme)

LOGGING = {
    ...  # no changes here
}

For more details, see Styles documentation. To inspect available theme keys and default values, issue

$ python -m rich.theme

and look for keys prefixed with log. or logging..

Outro

Note that rich is so much more than just coloured logging; go check it out:

$ python -m pip install rich
$ python -m rich

Specifically for the logging use case, check out the output of

$ python -m rich.logging

to see more rendering examples than in the screenshots.

Comments

10

Django already has support for color output through the 'DJANGO_COLORS' environment variable used for example when running the built in development server. Some person has noticed this and created a plug-and-play solution https://github.com/tiliv/django-colors-formatter; with that package on the project's python path my logging settings.py is as follow:

LOGGING = {
    'version': 1,
    'disable_existing_loggers': True,
    'filters': {
        'require_debug_false': {
            '()': 'django.utils.log.RequireDebugFalse'
        }
    },
    'formatters': {
        'verbose': {
            '()': 'djangocolors_formatter.DjangoColorsFormatter', # colored output
            'format': '%(levelname)s %(name)s %(asctime)s %(module)s %(process)d %(thread)d %(pathname)s@%(lineno)s: %(message)s'
        },
        'simple': {
            '()': 'djangocolors_formatter.DjangoColorsFormatter', # colored output
            'format': '%(levelname)s %(name)s %(filename)s@%(lineno)s: %(message)s'
        },
    },
     # omitting the handler 'level' setting so that all messages are passed and we do level filtering in 'loggers'
    'handlers': {
        'null': {
            'class':'django.utils.log.NullHandler',
        },
        'console':{
            'class':'logging.StreamHandler',
            'formatter': 'simple',
        },
        'mail_admins': {
            'filters': ['require_debug_false'],
            'class': 'django.utils.log.AdminEmailHandler',
            'formatter': 'verbose'
        }
    },
    'loggers': {
        '': { 
            'handlers': ['mail_admins', 'console'],
            'level': 'WARNING',
        },
    }
}

Sample console logging output using django-colors-formatter: Sample console logging output

2 Comments

Install via pip install git+https://github.com/tiliv/django-colors-formatter.git
This package seems abandoned. It hasn't received any attention in 5 years. Try using coloredlogs instead. It's a generic python package that works really well with Django.
5

I also wanted color output for the dev_appserver. I found the solutions here a little OTT (all I wanted was to make my logging.error() calls stand out. I ended up monkeypatching the logging module by dropping this in my main.py as a quick solution:

# monkey patch logger to dump ERRORs in red
import os
if os.environ['SERVER_SOFTWARE'].find('Development') >= 0:
    import logging
    old_error = logging.error
    def red_error(msg,*args,**kwargs):
        old_error("\033[22;31m%s\033[0;0m" % msg, *args, **kwargs)
    logging.error = red_error

This will only for on ANSI-color terminals.

Comments

4

Here is a sample formater:

class Formatter(logging.Formatter) :
    _level_colors  = {
      "DEBUG": "\033[22;32m", "INFO": "\033[01;34m",
      "WARNING": "\033[22;35m", "ERROR": "\033[22;31m",
      "CRITICAL": "\033[01;31m"
     };    

    def format(self, record):
        if(Formatter._level_colors.has_key(record.levelname)):
            record.levelname = "%s%s\033[0;0m" % \
                            (Formatter._level_colors[record.levelname],
                             record.levelname)
        record.name = "\033[37m\033[1m%s\033[0;0m" % record.name
        return logging.Formatter.format(self, record)    

You need to configure it, for example:

...
[formatters]
keys=console_formatter
...
[handler_console_handler]
class=StreamHandler
formatter=console_formatter
args=(sys.stdout,)

1 Comment

Why is \033 needed?
3

I used the coloredlogs package. Unlike DJANGO_COLORS, it's not specific to Django commands, and unlike django-colors-formatter, it's actively maintained.

I added a single line to my logging config, and now I get configurable coloured logs.

logging.config.dictConfig({
    ...
    'formatters': {
        'console': {
            # This line right here:
            "()": "coloredlogs.ColoredFormatter",
            'format': '%(asctime)s %(levelname)s [%(name)s:%(lineno)s] %(message)s',
        },
    },
    ...

Comments

2

The reset codes mentioned in the answer you linked to will work on a console in the local development server (but will likely take some tweaking - you'll have to chain it with the existing App Engine logging handler), but won't work in production, since in production log entries are output to an HTML page in your admin console.

You can, however, filter by log level in the admin console.

1 Comment

Thanks for answering, Nick. It's the chaining part that I'm having trouble with. :)
2

I don't believe that you should create a logger subclass just for this - airmind's answer is fine as far as creating a specialised Formatter and specifying its use on a StreamHandler. But there's no need for a logger subclass. In fact airmind's use of a logger class adds a handler to every logger created, which is not what you want.

The solution airmind gave only works for terminals which support ANSI escape sequences - are you sure that your console does support them?

1 Comment

Thanks Vinay. Yes, my terminal supports ANSI colors.
2

I wanted something similar, a quick way to highlight errors and warnings.

  • No external dependency for such a thing
  • Low code or no code, if possible
import logging

from django.utils.log import ServerFormatter


class LogFormatter(ServerFormatter):
    def format(self, record):
        msg = super().format(record)

        if record.levelno >= logging.ERROR:
            msg = self.style.ERROR(msg)
        elif record.levelno >= logging.WARNING:
            msg = self.style.WARNING(msg)

        return msg

In logging configuration:

   "formatters": {
        "console": {
            "()": "myapp.utils.logging.LogFormatter",
            "format": "%(asctime)s [%(name)s] %(levelname)s: %(message)s",
            "datefmt": "%Y-%m-%d %H:%M:%S",
        },
    },

Originally, the ServerFormatter highlights the message only, so the timestamp, logger name etc. are not highlighted. This LogFormatter highlights the entire line and overrides any highlights done by the ServerFormatter.

Comments

1

Install colorlog

(don't forget to put colorlog in INSTALLED_APPS)

create 'colored' in your formatter

 'colored': {
        '()': 'colorlog.ColoredFormatter',
        'format': "%(log_color)s %(levelname)-8s %(asctime)s %(module)s %(reset)s %(blue)s%(message)s",
    }

full example of logger in settings.py

LOGGING = {
'version': 1,
'disable_existing_loggers': False,
'formatters': {
    'colored': {
        '()': 'colorlog.ColoredFormatter',
        'format': "%(log_color)s %(levelname)-8s %(asctime)s %(module)s %(reset)s %(blue)s%(message)s",
    }
},
'handlers': {
    'console': {
        'class': 'logging.StreamHandler',
        'formatter': 'colored'
    },
},
'root': {
    'handlers': ['console'],
    'level': 'WARNING',
},
'loggers': {
    'django': {
        'handlers': ['console'],
        'level': os.getenv('DJANGO_LOG_LEVEL', 'INFO'),
        'propagate': False,
     },
  },
}

enter image description here

Comments

0

Here is my solution for Django with colorlog. It just colors simple django messages as they are. You only need to put it in your settings.py.

pip install colorlog
LOGGING = {
    'version': 1,
    'disable_existing_loggers': False,
    'formatters': {
        'special': {
            '()': 'colorlog.ColoredFormatter',
            'format': '%(log_color)s[%(asctime)s] %(message)s'
        }
    },
    'filters': {
        'require_debug_true': {
            '()': 'django.utils.log.RequireDebugTrue',
        },
    },
    'handlers': {
        'console': {
            'level': 'INFO',
            'filters': ['require_debug_true'],
            'class': 'logging.StreamHandler',
            'formatter': 'special'
        }
    },
    'loggers': {
        'django': {
            'handlers': ['console'],
            'propagate': True,
        }
    }
}

Screen

Comments

0

In order to reuse standard Django's colorized formatter (no extra 3rd party libs needed) I have copied and adapted django.utils.log.DEFAULT_LOGGING configuration.

For instance:

LOGGING = {
    'formatters': {
        "django.server2": {
            # NOTE: coloring based on django/utils/log.py :: DEFAULT_LOGGING
            "()": "django.utils.log.ServerFormatter",
            "format": "%(levelname)-7s %(asctime)s %(message)s",
        },
        ...
    },
    'handlers': {
        "django.server2": {
            "level": "INFO",
            "class": "logging.StreamHandler",
            "formatter": "django.server2",
        },
        ...
    },
    'loggers': {
        'django.server': {
            'handlers': ["django.server2"],
            'level': 'INFO',
            "propagate": False,
        },
        ...
    },
}

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.