My favorites | Sign in
Project Logo
             
Search
for
Updated Feb 15, 2009 by frasern
Labels: Featured
Overview  
A very brief overview of django-logging.

Introduction

It is really useful to know what your Django project is doing, especially when debugging.

If you're running Django via runserver, then you can simply use print statements and they will appear in the console. If you're running via mod_python, then you can output the print statements to the web server's error log by redirecting to stdout (print >>sys.stdout).

While useful, these both have their limitations. Python had a built-in logging system, so django-logging was created to allow it to be easily used within Django projects.

What's new

Installation

It's assumed that you have at least a basic knowledge of Django, Python and Subversion.

The source code for django-logging is stored in a Subversion repository, so to start with, you'll need to check out a copy, making sure that you put it somewhere on your PYTHONPATH:

svn co http://django-logging.googlecode.com/svn/trunk/djangologging/ djangologging

Step 1. Add the django-logging middleware to your Django project's settings file:

MIDDLEWARE_CLASSES = (
    ...
    'djangologging.middleware.LoggingMiddleware',
    ...
)

The order of MIDDLEWARE_CLASSES is important: the django-logging middleware must come after any other middleware that encodes the response's content (such as GZipMiddleware).

Step 2. The django-logging middleware will only operate if your IP address is in the INTERNAL_IPS setting, so you will have to set this appropriately. If you're working locally, you may need to set this to something like:

INTERNAL_IPS = ('127.0.0.1',)

Step 3. As an additional security measure, the django-logging middleware will only add its output to HTML pages when:

  • LOGGING_OUTPUT_ENABLED is set to True; or
  • LOGGING_OUTPUT_ENABLED is undefined and DEBUG is set to True.

That's it! (See the Configuration section below for information on the various configurable options.)

Usage

django-logging uses the standard Python logging module, so simply import it and log whatever messages you like. For example:

import logging

logging.debug('This is a sample debug message')
logging.info('This is a sample informational message')
logging.warn('This is a sample warning message')
logging.error('This is a sample error message')
logging.critical('This is a sample critical message')

When you view a page in your browser, any log messages that were created during the processing of your request will appear at the bottom of the page.

One key feature of all this is that because this is using the standard logging framework, you can add other handlers to do other things, such as logging ERROR and CRITICAL level messages to a file. This could be expecially useful on a production server.

There are a number of sites which provide some useful information on using the logging module, including:

How it works

A custom log handler is created that buffers all messages logged on a per-thread basis. This equates to a per-request basis within Django.

After the request has been processed and response generated, the middleware checks if the response is a HTML page (i.e. has a Content-Type header of text/html). If so, it writes the request's log messages into the HTML document. Extra HTTP headers are also added to indicate that the rewritten page should not be cached.

Note that unlike the built-in Django debug pages, django-logging does not sanitise any of the content; that is if you log any sensitve information (such as passwords), this will be displayed verbatim in clear text.

Configuration

django-logging should just work "out of the box", but you can configure the following options in your Django settings file:

Variable Default Description
LOGGING_INTERCEPT_REDIRECTS False Setting this to True will cause django-logging to intercept any HTTP redirects and output an interim page to allow you to see any messages that would have otherwise been lost.
LOGGING_LOG_SQL False Setting this to True and having settings.DEBUG set to True will cause django-logging to log all SQL queries made through django.db.connection to be logged at a custom level, SQL. If installed, the Pygments syntax highlighter will be used to pretty print the SQL.
LOGGING_OUTPUT_ENABLED settings.DEBUG Setting this to True will cause django-logging to be active and write the log data into outputted HTML pages. Setting this to False will prevent django-logging from outputting the log data.
LOGGING_SHOW_METRICS True Setting this to True will cause django-logging calculate and display basic metric information, such as request duration and number of database queries.
LOGGING_TEMPLATE_DIR Auto determined If you wish to use your own templates, set this to the full path to the directory containing your templates. This directory should contain your own versions of all the standard template files: logging.html, logging.css and redirect.html.
LOGGING_REWRITE_CONTENT_TYPES ('text/html',) If you wish django-logging to add its output to responses with a Content-Type other than text/html, set this to a sequence of content type strings. For example, to enable rewriting of both HTML (Content-Type: text/html) and XHTML (Content-Type: application/xhtml+xml) pages, set this to ('text/html', 'application/xhtml+xml').

Output suppression

If there is a particular view that generates log messages that should not be appended to the page, you can use the suppress_logging_output decorator. For example:

from djangologging.decorators import suppress_logging_output

@suppress_logging_output
def some_view(request):
    ...

If you're using AJAX to request a page, you may not wish django-logging to append its output to the response. Most Javascript libraries will add the X-Requested-With: XMLHttpRequest to AJAX requests, so you can use the SuppressLoggingOnAjaxRequestsMiddleware to stop log messages from being outputted. This middleware should be installed after the main logging middleware:

MIDDLEWARE_CLASSES = (
    ...
    'djangologging.middleware.LoggingMiddleware',
    'djangologging.middleware.SuppressLoggingOnAjaxRequestsMiddleware',
    ...
)

Future Features

Some things I'd like to add when I get the time:

Feedback

If you spot any bugs, have suggestions for improvement or ideas for future features, please use the issue tracker.

If you wish to email, my details are on my own site's contact page.


Comment by pgugged, Jun 19, 2008

Hi

django-logging middleware doesn't work with Django version 0.97-pre-SVN-7698, it doesn't append the html info like it should

Comment by frasern, Jun 22, 2008

I've looked into this and as a result noticed that the documentation of was slightly, but very significantly, wrong. Within MIDDLEWARE_CLASSES, the django-logging middleware must come after any other middleware that encodes the response's content (such as GZipMiddleware).

I have updated the documentation (r23) to correct this.

Can you try again in your environment? If you are still experiencing problems, please raise a new issue with as much detail as you can.

Comment by pgugged, Jun 22, 2008

It is working now!, thanks

Comment by tobias.mcnulty, Jul 02, 2008

Is there any way to use django-logging to log messages to a file (or e-mail) on a production server? Thanks!

Comment by tobias.mcnulty, Jul 02, 2008

Doh. I just found the part of the docs above that say exactly what I'm looking for. Nevermind.

Comment by russelljryan, Sep 27, 2008

You and the django-debug-bar should join projects! It would be really useful to have the log output as a panel of the debug-bar.

Thanks for this, it's very useful.

RJ Ryan

Comment by dan.ferrante, Oct 05, 2008

you guys need to advertise more, i got halfway into making a whole template tag/filter library to output queries and such before i found this :). Nice job, works great with django 1.0. One comment, you should add that you have to explicitly define INTERNAL_IPS in settings, took me a few minutes to figure out that it does not have a default value within django.

Comment by ransom1982, Nov 09, 2008

I found that with INTERNAL_IPS set to '127.0.0.1' that logging was always enabled whether or not DEBUG was true or LOGGING_OUTPUT_ENABLED was false.

Comment by carlzlhu, Jan 20, 2009

Absolutely amazing job, guys. For folks who need to use apache for a development server, this is by far the best way to work. Using 'tail -f apache.log' and figuring out how to print to the apache log is a fairly miserable experience compared to this. Again, thank you so much.

Comment by chyld.medford, Mar 05, 2009

thanks for making such an excellent tool!

Comment by pvilas, Mar 13, 2009

Does not work if the encoding of the source file is # -- coding: utf8 --

Thanks a lot!

Comment by mark.hel...@gmail.com, Apr 02, 2009

Thanks for this, love it!

Comment by mark.hel...@gmail.com, Apr 03, 2009

'lo, do I have to do anything special to get this working for the admin interface? got it all setup as required but I only ever see "No log entries" underneath the "Request Log" bar at the bottom of the page :(

Comment by roscoedesign, Apr 13, 2009

I'm using Django 1.0-final-SVN-9017 and I have a simple view that merely loads a template and the RequestContext?(request) before I return the render_to_response I am calling import logging and then logging.debug('test') and when I load the page it outputs this error:

IOError at /my/views/url/path/ could not get source code

Any idea what's happening?

Comment by ref...@gmail.com, Jun 16, 2009

I've branched the code into bitbucket: http://bitbucket.org/refack/djangologging/ Adding several bug fixes (such as cookie handling & the infamous IOError), and also some new features, like request.session content logging, and Traceback.


Sign in to add a comment
Hosted by Google Code