20May

Fixing an omission from Django’s simplejson: iterators, generators, functors and closures

Posted by Elf Sternberg as javascript, programming, python, Uncategorized

Most people encountering the Ajax components of Django for the first time usually encounter this pairing:

from django.utils import simplejson
...
return HttpResponse(simplejson.dumps(some_python_obj))

For most applications, this is enough.  But I’m especially fond of iterators, generators, functors and closures, dense components that express one thought well: the structure of a tree, or the rows of a database, to be sent to the browser.  The routine dumps() doesn’t understand any of those things, but with a simple addition, you can plug them into your code and be on your way without headache.  Dumps() just calls JSONEncoder(), and JSONEncoder has a routine for extending its functionality.

The routine is to override a method named default() (why “default?” I have no idea) and add the object types and signatures you want to send to the browser.  Normally, this exists for you to provide custom “object to JSON” handlers for your objects, but there’s nothing custom about iterators, generators, functors and closures.  They are native Python objects.

from django.utils.simplejson.encoder import JSONEncoder

class ExtJsonEncoder(JSONEncoder):
    def default(self, c):
        # Handles generators and iterators
        if hasattr(c, '__iter__'):
            return [i for i in c]

        # Handles closures and functors
        if hasattr(c, '__call__'):
            return c()

        return JSONEncoder.default(self, c)

This exploits the OO nature of Python code: everything is an object, and I’m just checking to see if the object passed in has the interface needed to treat it like an iterator or a function.  If it’s an iterator, iterate it; if it’s a function, call it.
Normally I like my code to be sensible, so I devolve this into a function, just like dumps():

from django.utils import simplejson

def json(s, **kw):
    kw.update({'cls': ExtJsonEncoder})
    return simplejson.dumps(s, **kw)

To show this in use, I’ll use the Tag() tree class I’ve been working on this week.  First, here’s the “Django Tree to Dojo Tree JSON” as a functor (a functor is a class that includes a method named __call__() in its interface, and a reference to an object of that class can be called like a function):

class TreeToJson:
    def __init__(self, roots):
        self.roots = roots

    def _index(self, nodes):
        def _item(node):
            t = { 'id': node.id,
                  'name': node.name,
                  'top': node.is_root()}
            if not node.is_leaf():
                t.update({'children': self._index(node.get_children())})
            return t

        return [_item(node) for node in nodes]

    def __call__(self):
        return self._index(self.roots)

print json(TreeToJson(Tag.get_root_nodes())

The class TreeToJson completely encapsulates the process of converting a django-treebeard tree into a dojo tree structure here. As an alternative, here it is as a generator:

def treetojson(roots):
    def _item(node):
        t = { 'id': node.id,
              'name': node.name,
              'top': node.is_root()}
        if not node.is_leaf():
            t.update({'children': _tree(node.get_children())})
        return t

    def _tree(nodes):
        for node in nodes:
            yield _item(node)

    return _tree(roots)

print json(treetojson(Tag.get_root_nodes()))

Again, the function treetojson completely encapsulates the process of converting a django-treebeard tree into a dojo tree.  Both of these work quite nicely, spewing out a Javascript Object ready to be eval’d and processed on the client.

Now, obviously, my implementation of ExtJsonEncoder has its opinions: it prioritizes iterators over functions.  But I’ve been using this implementation with this priority with Webware and EXT-JS (and a handwritten version of simplejson) and I have yet to encounter a case where a class had both __iter__() and __call__() defined or needed the latter prioritized.

6 Responses to Fixing an omission from Django’s simplejson: iterators, generators, functors and closures

Jon Gales

May 20th, 2009 at 2:01 pm

While this is very handy (I just had to do the same thing last week), it’s inaccurate to call it “Django’s simplejson”. simplejson is a third party library that just happens to be included in Django since it’s full stack and tries to bundle all the dependencies.

http://code.google.com/p/simplejson/

Elf Sternberg

May 20th, 2009 at 2:11 pm

That’s true. Someone else commented on that on my personal blog, and I pointed out that this technique works with Pylons and Webware just as readily. It’s just that I was doing this in a Django-ish context. I should amend it to mention that this works with any implementation of simplejson.

Jon Gales

May 20th, 2009 at 5:57 pm

No biggy :) . Also it looks like you have a typo:

{‘cls’, ExtJsonEncoder}

That’s invalid, it should be a colon like so:

{‘cls’: ExtJsonEncoder}

Otherwise it works like a charm.

Elf Sternberg

May 20th, 2009 at 6:00 pm

Thanks, fixed it. Dammit, I know I fixed that in the test code, too, so how it wound up in the post… Sigh. So annoying.

Rogério Carrasqueira

September 20th, 2010 at 3:39 pm

Hi! I’m trying to implement your code on my application, putting ExtJsonEnconder and json on my utils file and treetojson at my views.py and I’m getting this error:

__init__() got an unexpected keyword argument ‘use_decimal’

Do you have any idea what is happening?

Cheers

Rogério

Elf Sternberg

September 22nd, 2010 at 9:22 am

I really couldn’t tell you; it appears that you’re passing the object you want encoded to the ExtJsonEncoder incorrectly. Are you doing anything odd like using an asterisk to pass values?

Comment Form

Recent Comments