20Nov

Python: Decorators With Arguments (with bonus Django goodness)

Posted by Elf Sternberg as django, programming, python

For the past few days, this has been bugging the Hell out of me and I finally decided today to knuckle down and figure out how the hell Python decorators-with-arguments work.

Basic Decoration

The basics: A python function takes arguments, performs a task, and returns a value. A decorator takes as its argument a function, and returns another function with different semantics. It has “decorated” the function with pre-call and post-call behaviors and conditions.

The biggest (and hardest) part for many beginning programmers to grasp is this: a function name is just a variable.  A function is just an object; when you call a function, you’re (1) dereferencing the variable functionname, (2)  checking that the referenced object is callable, (3) passing that object arguments and (4) invoking a new execution frame, which in turn builds a context and attempts to run the referenced object with the arguments.   (That context, by the way, persists for all functions defined within it at each execution pass; this is how closure works, and decorators are highly closure-dependent.)

The classic example:

def decorator(f):
    def wrapper(*args, **kwargs):
        print "This begins decoration!"
        f(*args, **kwargs)
        print "This ends decoration!"
    return wrapper

def example(argument):
    print "This is the argument:", argument

print "Example without decoration:"
example("one!")

print "\nExample with decoration:"
example = decorator(example)
example("two!")

As you can see, in the “Example with decoration” part, we passed the example function object to our decorator function, which returned a new function that, as part of its mission, called the example function object, but also prefixed and postfixed that call with some other expressions– in this case, simple print statements.

Basic Decorators with signature preserved

One of the problems with the above is that the signature of the original function is not preserved. Instead, we get the name of the inner containing function providing the decorations. If we had an error occur in “example,” we would see the error reported as:

File "demo.py", line 4, in wrapper

This isn’t what we want at all.

The python module functools provides a routine, wraps, that decorates the return value of a decorator with the proper signature. Add the import line and replace decorator with this:

from functools import wraps

def decorator(f):
    def wrapper(*args, **kwargs):
        print "This begins decoration!"
        f(*args, **kwargs)
        print "This ends decoration!"
    return wraps(f)(wrapper)

The function wraps returns a function prepared with handle the signature of f, which then takes the function wrapper, and decorates that function with the signature of f, returning a new function handle. Now, your tracebacks will contain the correct name of the function, whatever f happens to be, that caused the exception.

Decorators with Arguments

This took me forever to wrap my head around. This syntax:

@decorate
def function(a, b):
    ....

is functionally equivalent to:

function = decorate(function)

But, this syntax:

@decorate("Additional info")
def function(a, b):
    ...

is functionally equivalent to:

function = decorate("Additional info")(function)

Which requires an extra level of indirection in your decorator.   A decorator with arguments must return the real decorator that, in turn, decorates the function.

So here’s where the Django goodness comes in.  Django has a function, render_to_response, that takes two required arguments and one optional.  The required arguments are the HTML Template to render, a dictionary of values to substitute on the page; the optional argument is the context, which provides extra information not necessarily directly related to the current command cycle, but necessary for the page to make sense.  An example of this kind of extra information would be the user’s identity, and current statistics related to his usage, but not to the current task.

In some ways, what I’m about to do is go over the same ground as django.views.generic.list_detail.object_detail, but I think my way is somewhat more interesting, and is illustrative to the task at hand. What I’d like is to encapsulate my business logic in one function, and the details of rendering it could then be added in later via decoration. Here’s a very simple example:

@render('cards/home.html')
def home(request):
    return {'cards ':Card.objects.all()}

The business logic is simple: the home page for this application returns all the cards. And the decoration is equally simple: We’re literally going to “decorate” this data with the HTML for the home page.

So, we need a function that wraps Django’s render_to_response, decorating it with the template name and the RequestContext object (that provides all the miscellaneous information most templates in Django need). The outermost decorator ultimately needs to return a function that does this while preserving the template name in a closure. And we want to wrap our innermost function in the name of the function we pass in. Here’s the entirety of it:

from django.template import RequestContext
from functools import wraps
from django.shortcuts import render_to_response

def render(template_name):
    def inner_render(fn):
        def wrapped(request, *args, **kwargs):
            return render_to_response(template_name,
                                       fn(request, *args, **kwargs),
                                       context_instance = RequestContext(request))
        return wraps(fn)(wrapped)
    return inner_render

When render is invoked as above, it encloses the template_name field and returns a decorator function prepared to handle the function-to-be-decorated, fn. That function, fn, in turn is transmogrified from one that returns a dictionary into one that returns a fully functional Django-rendered HTML page, and has the correct signature in case something goes wrong (and believe me, Django needs all the help it can get tracking down errors when something goes wrong).

This is plenty of old ground here, but it’s useful for me to go through exercises like this to understand what’s going on inside python’s “modern” decorator syntax.   There’s also much grousing about why the standard template renderers always supply the context, but render_to_response does not, and plenty of suggested solutions.  This is one.

8 Responses to Python: Decorators With Arguments (with bonus Django goodness)

Rob

November 21st, 2009 at 5:23 am

You pass the results of fn() directly to render_to_response. If fn() returns a HttpResponseRedirect then you will get an error because render_to_response is expecting a dict as its second argument. Instead you should get the response of fn() and check to see if it is a response object; if it is then return it else pass it to render_to_response

Elf Sternberg

November 24th, 2009 at 9:18 am

Rob, the point is that fn() isn’t going to return an HttpResponseRedirect; the entire API is set up so that fn() returns a python dictionary (or dictionary-like; remember duck-typing) object, and the developer decorates that response with the appropriate handler. This way, the rendering and the business logic are separate; the rule is simple: business modules return dictionaries of meaningful data, not the arbitrary bytestream of an HTML page.

Jeethu Rao

November 28th, 2009 at 5:03 am

Rob is right. How hard is it to check isinstance(r,HttpResponse) and return it directly ? I feel your argument about seperating the rendering and the business logic is just a strawman. Aren’t properly written applications not supposed to be able to redirect ?

Exploring Python @decorators | Differential Progression

May 28th, 2011 at 10:02 am

[...] as usual, one goes off and does some research. My answer was found here in a very useful post by Elf Sternberg. The example on the linked page has a bit more detail than [...]

jonEbird » Python Memoize Decorator with TTL Argument

February 7th, 2012 at 7:02 pm

[...] page. (I also spent some time re-reading Bruce Eckel’s Decorator Arguments writeup as well as Elf Sternberg’s Decorators With Arguments [...]

Django Custom View Decorator with Arguments | Samos IT Blog

February 23rd, 2012 at 10:52 am

[...] I noticed there weren’t a lot of guides about how to write a decorator for Django which takes arguments. I based my code on this tutorial, it’s a really great guide which helps you understand Python decorators at a new level: Amazing insight about Python decorators [...]

Taras

February 25th, 2012 at 11:49 pm

+1

Duncan Payne

August 12th, 2013 at 4:07 am

Great in-depth article on decorators. I have based some of what I have done on your post. I am using a kind-of login_required decorator, so once everything has been verified, I have still needed to return fn(request, *args, **kwargs) at the end of my equivalent wrapped function.

Thanks for the pointers though, and the explanation.

Comment Form

Recent Comments