My favorites | Sign in
Project Home Downloads Wiki Issues Source
Search
for
DjangoFullSerializers  
Documentation and examples for the Wad of Stuff Django Full Serializers
Featured, Phase-Implementation
Updated Mar 6, 2011 by mattimus...@gmail.com

Introduction

The wadofstuff.django.serializers python module extends Django's built-in serializers, adding 3 new capabilities inspired by the Ruby on Rails JSON serializer. These parameters allow the developer more control over how their models are serialized. The additional capabilities are:

  • excludes - a list of fields to be excluded from serialization. The excludes list takes precedence over the fields argument.
  • extras - a list of non-model field properties or callables to be serialized.
  • relations - a list or dictionary of model related fields to be followed and serialized.

Serialization Formats

The module currently support serializing to JSON and Python only.

Source

The latest stable release for the serialization module can be obtained by:

  • Running easy_install wadofstuff-django-serializers

The latest development source can be obtained here:

Examples

Project Settings

You must add the following to your project's settings.py to be able to use the JSON serializer.

SERIALIZATION_MODULES = {
    'json': 'wadofstuff.django.serializers.json'
}

Backwards Compatibility

The Wad of Stuff serializers are 100% compatible with the Django serializers when serializing a model. When deserializing a data stream the the Deserializer class currently only works with serialized data returned by the standard Django serializers.

>>> from django.contrib.auth.models import Group
>>> from django.core import serializers
>>> print serializers.serialize('json', Group.objects.all(), indent=4)
[
    {
        "pk": 2,
        "model": "auth.group",
        "fields": {
            "name": "session",
            "permissions": [
                19
            ]
        }
    }
]

Excludes

>>> print serializers.serialize('json', Group.objects.all(), indent=4, excludes=('permissions',))
[
    {
        "pk": 2,
        "model": "auth.group",
        "fields": {
            "name": "session"
        }
    }
]

Extras

The extras option allows the developer to serialize properties of a model that are not fields. These properties may be almost any standard python attribute or method. The only limitation being that you may only serialize methods that do not require any arguments.

For demonstration purposes in this example I monkey patch the Group model to have a get_absolute_url() method.

>>> def get_absolute_url(self):
...     return u'/group/%s' % self.name
...
>>> Group.get_absolute_url = get_absolute_url
>>> print serializers.serialize('json', Group.objects.all(), indent=4, extras=('__unicode__','get_absolute_url'))
[
    {
        "pk": 2,
        "model": "auth.group",
        "extras": {
            "get_absolute_url": "/group/session",
            "__unicode__": "session"
        },
        "fields": {
            "name": "session",
            "permissions": [
                19
            ]
        }
    }
]

Relations

The Wad of Stuff serializers allow you to follow related fields of a model to any depth you wish and serialize those as well. This is why it is considered a "full serializer" as opposed to Django's built-in serializers that only return the related fields primary key value.

When using the relations argument to the serializer you may specify either a list of fields to be serialized or a dictionary of key/value pairs. The dictionary keys are the field names of the related fields to be serialized and the values are the arguments to pass to the serializer when processing that field.

>>> print serializers.serialize('json', Group.objects.all(), indent=4, relations=('permissions',))
[
    {
        "pk": 2,
        "model": "auth.group",
        "fields": {
            "name": "session",
            "permissions": [
                {
                    "pk": 19,
                    "model": "auth.permission",
                    "fields": {
                        "codename": "add_session",
                        "name": "Can add session",
                        "content_type": 7
                    }
                }
            ]
        }
    }
]

Only serializing a particular field of a relation

>>> print serializers.serialize('json', Group.objects.all(), indent=4, relations={'permissions':{'fields':('codename',)}})
[
    {
        "pk": 2,
        "model": "auth.group",
        "fields": {
            "name": "session",
            "permissions": [
                {
                    "pk": 19,
                    "model": "auth.permission",
                    "fields": {
                        "codename": "add_session"
                    }
                }
            ]
        }
    }
]

Serializing a relation of a relation

>>> print serializers.serialize('json', Group.objects.all(), indent=4, relations={'permissions':{'relations':('content_type',)}})
[
    {
        "pk": 2,
        "model": "auth.group",
        "fields": {
            "name": "session",
            "permissions": [
                {
                    "pk": 19,
                    "model": "auth.permission",
                    "fields": {
                        "codename": "add_session",
                        "name": "Can add session",
                        "content_type": {
                            "pk": 7,
                            "model": "contenttypes.contenttype",
                            "fields": {
                                "model": "session",
                                "name": "session",
                                "app_label": "sessions"
                            }
                        }
                    }
                }
            ]
        }
    }
]

Excluding a field from a relation of a relation

>>> print serializers.serialize('json', Group.objects.all(), indent=4, relations={'permissions':{'relations':{'content_type':{'excludes':('app_label',)}}}})
[
    {
        "pk": 2,
        "model": "auth.group",
        "fields": {
            "name": "session",
            "permissions": [
                {
                    "pk": 19,
                    "model": "auth.permission",
                    "fields": {
                        "codename": "add_session",
                        "name": "Can add session",
                        "content_type": {
                            "pk": 7,
                            "model": "contenttypes.contenttype",
                            "fields": {
                                "model": "session",
                                "name": "session"
                            }
                        }
                    }
                }
            ]
        }
    }
]
Comment by jumi...@gmail.com, Feb 26, 2009

Extremely helpful - thank you.

Comment by ishur...@gmail.com, Mar 11, 2009

Brilliant serializer, proven very useful. Thanks a lot

Comment by tony.mcd...@gmail.com, Apr 5, 2009

The 'extras' facility has just saved me heaps of time - I had no idea why it wasn't being serialized, and fortunately found your post.

many thanks! tone

Comment by tony.mcd...@gmail.com, Apr 11, 2009

I've passed on this link to the guys doing great work at the Washington Times <http://opensource.washingtontimes.com/> as they (Justin) have been working on a terrific django app called django-apibuilder which is a beautiful fit for this serializer.

Comment by justqu...@gmail.com, Apr 18, 2009

yes, the wad is right now being used in the development branch of the django-apibuilder

Comment by Flashing...@gmail.com, Apr 30, 2009

Short question about the possibilities to serialize fields inherited by subclassing a django model. There's basically something like this:

from django.db import models

class BaseModel(models.Model):
    name = models.CharField()
    
class InheritModel(BaseModel):
    content = models.TextField()
    
class OtherModel(BaseModel):
    date = models.DateTimeField()
     

serializing either InheritModel? or OtherModel? does not yield serialization of the name field of BaseModel?, and I couldn't find how to make the link to the related field on a quick glance. Tried both of these options

serialize('json',InheritModel.objects.all(), relations = {'basemodel':{'fields':('name',)}}) 
serialize('json',InheritModel.objects.all(), relations = {'basemodel_ptr':{'fields':('name',)}}) # which is the fields name specified via InheritModel._meta

Any suggestions on how to solve, or would it need to add some more code to the Serializer class?

Cheers

Comment by Flashing...@gmail.com, Apr 30, 2009

PS: BaseModel? is not an abstract = True class.

Comment by Flashing...@gmail.com, Apr 30, 2009

Well, looked a bit more into it and figured out, that django.db.models.fields.OneToOneField , which handles inheritance, is set to serialize = False on inheritance which skipped the serialization. I've added another loop which iterates over <obj>._meta.parents and updates self._fields with the values from the parent's object fields. You'll find the the svn diff in the issues. Hope I was able to enhance this awesome serializer. :)

Cheers,

Comment by sergio.b...@gmail.com, Jul 31, 2009

How to i serialize infinite model Tree, like this:

class Item(models.Model):

item_name: models.CharField?(max_length=30) parent_item: models.ForeignKey?(self)

thank you

Comment by project member mattimus...@gmail.com, Aug 1, 2009

You have to explicitly specify to depth you want to serialize to. e.g this will serial 3 relations deep.

>>> print serializers.serialize('json', Group.objects.all(), indent=4,
    relations={
        'parent_item':{
            'relations':{
                'parent_item':{
                    'relations':('parent_item',)
                }
            }
        }
    })
Comment by delwy...@gmail.com, Aug 3, 2009

Brilliant serializer, using it in a few django apps. What is the best way to use it with the google app engine?

Thanks in advance.

Comment by project member mattimus...@gmail.com, Aug 3, 2009

Hi,

Sorry I haven't tried it with GAE. If you can get it working let me know and pass along any patches.

Comment by xarali...@gmail.com, Nov 5, 2009

I am having little trouble when trying to serialize a model created by multi-table model inheritance.

I have tracked down the source to the fact that django marks the link from model to it's parent (OneToOneField?) with serialize=False. This results in not serializing the field by the serializer.

I need to serialize the fields from the parent as well.

Do you have some hint how to prevent Django from marking it up like this?

Comment by fuzzyman, Nov 13, 2009

In the extras documentation you say: These properties may be almost any standard python attribute or method.

But it seems to me (from experimentation) that they can't be any attribute, they have to be a method that takes no arguments.

In addition if this method returns anything other than a string then the object is serialized as a string. I need to serialize a list of dictionaries and this is coming out as a string in the json.

This makes me a sad bunny as I then have to serialize it myself. :-(

Comment by johnvmcc...@gmail.com, Nov 17, 2009

Great work on the serializer, thank you very, very much!

Comment by jameskhe...@gmail.com, Nov 30, 2009

Using this with a pyjamas front end - this has saved me a lot of time! Thank you very much. What I like is that it has just enough flexibility to be useful but not too much that it takes a long time to figure out.

Comment by mgsb...@gmail.com, Feb 19, 2010

Excellent work. It helped in greatly in handling fully loaded objects in case of ManyToMany? relation.

When are you planing to provide the similar support on XML serialization?

Comment by eugene.k...@gmail.com, May 7, 2010

I'm trying to use DjangoFullSerializers in 'python' mode, but it seems it's not producing the appropriate data structures for relations?

Comment by project member mattimus...@gmail.com, May 9, 2010

@eugene. please post some code and output from it as you've not really given me much to work with. Also, please open it as an issue rather than just a comment.

Comment by eugene.k...@gmail.com, May 10, 2010

Ok, thanks, issue submitted: http://code.google.com/p/wadofstuff/issues/detail?id=12 . I wasn't sure if this was a bug or per design.

Comment by floris...@gmail.com, Jul 19, 2010

Can extras mark non-field properties of related objects(through a FK) to be serialized ? I've tried this but have been unsuccessful so far.

Use case(if Group.permissions were a FK to Permission)

groups = Group.objects.all()
for g in groups:
    g.permissions.property = "Serialize me!"
print serializers.serialize('json', groups, indent=4, relations=('permissions',),extras={'permissions' : { 'extras' : ('property',)} })

Expected output(per group object):

[
    {
        "pk": 2,
        "model": "auth.group",
        "fields": {
            "name": "session",
            "permissions": [
                {
                    "pk": 19,
                    "model": "auth.permission",
                    "extras" : {
                       property : "Serialize me!"
                     },
                    "fields": {
                        "codename": "add_session",
                        "name": "Can add session",
                        "content_type": 7
                    }
                }
            ]
        }
    }
]
Comment by project member mattimus...@gmail.com, Jul 20, 2010

@florisvda This will probably do what you want:

groups = Group.objects.all()
for g in groups:
    g.permissions.property = "Serialize me!"
print serializers.serialize('json', groups, indent=4, relations={'permissions':{'extras':('property',)}})
Comment by vietnhi1...@gmail.com, Oct 12, 2010

Hi all, How can you actually perform an extras nested in another extras ? For ex: ... extras =('getA') --> then getA is performing another serialisation which has another extras (for ex: extras('getB'))

Comment by geert.pa...@gmail.com, Mar 17, 2011

I am new to Django and started with 1.3 so maybe I am doing something wrong. But in 1.3 extras only works if you pass in a list of extra fields to include. Was this always the case? The example shows a tupel.

Comment by project member mattimus...@gmail.com, Mar 21, 2011

@geert: I haven't tested with Django 1.3 yet. Can you create a new issue with some example code, including models, for what you are trying to do and I will test it out?

Comment by subraman...@gmail.com, May 29, 2011

Lets say I have the following two django (1.3) models

from django.db import models class Patient(models.Model):

name = models.CharField?('Name', max_length=50)
class Address(models.Model):

town = models.CharField?('Town/Village', max_length=50) patient = models.OneToOneField?(Patient, related_name='address')

Now when i try to serialize a Patient model instance to JSON, the resulting JSON string does'nt have the address details with it (it's unable to traverse through the reverse direction of One-to-one relation)

This happens event if I use select_related('address') to populate the address object into the cache. i.e.

from django.core import serializers >>> print serializers.serialize('json',Patient.objects.select_related('address').get(id=1)?)

Is there a way to get around this problem?

Comment by jgabriel...@unitec.edu, Jun 16, 2011

Incredible solution, 3 well almost 4 hours breaking my head trying to figure out why it didnt serialize querysets like this Model.objects.values('id','name') :S but this post was the solution

Comment by adn....@gmail.com, Aug 2, 2011

My question is how to access a certain date? Example I want to get the vehicle registration with id = 50 and coincides with the date 08.02.2011

But I do only this: I get the records which match the id of the vehicle, but not how to access the date I wanted to do so: Vehicle.objects.filter (id = IDV, posi.fecha = date) but is not valid

serializers.serialize result = ('json', Vehicle.objects.filter (id = IDV), indent = 4, relations = ('posi'))

class Position (models.Model): Latitude = models.CharField? (max_length = 30, null = True) longitude = models.CharField? (max_length = 30, null = True) fecha = models.DateField? date = (null = True)

class Vehicle (models.Model): models.ManyToManyField? posi = (Position) name = models.CharField? (max_length = 30, null = True) description = models.CharField? (max_length = 30, null = True)

Help me please, thanks

Comment by pgath...@gmail.com, Sep 8, 2011

Great product, thanks lots. But i have been wondering, how would you install it in django webfaction?

Comment by geert2...@gmail.com, Sep 19, 2011

hi we have been using this in one of our projects since november -- but an update to 1.1.0 didn't go well. I can't say it broke our code, because what happened was quite strange. Let me explain.

We were using it to produce json for related_names like thus:

p = Project.objects.filter(clientid=client_id) if not project_id else get_list_or_404(Project, id=project_id, clientid=client_id) data = serializers.serialize("json", p, indent=4, relations=('image_files', 'video_files', 'illustrations', 'solid_colors',))

where the relations are reverse relations, as in:

class Video(models.Model):

active = models.BooleanField?(default=True, help_text=("Set as active")) sort = models.CharField?(max_length=50, default='video', ) show_on_project = models.BooleanField?(default=False, help_text=("Show this video on project page")) name = models.CharField?(max_length=255, blank=True, default='') project = models.ForeignKey?(Project, related_name="video_files")
.....

Worked perfectly.

But for some reason this stopped working after upgrading to 1.1.0. No error, but just stopped working. No relations at all were seen. But the strangest thing, downgrading back to 1.0.0. didn't help. Only after I literally copied a version 1.0.0 (date created Nov 2010) from my own machine to the production server did I get the feature back.

Anyone know what is going on here? Thanks! (and apart from today's experience -- its great!!!!)

Geert

Comment by DrMarti...@gmail.com, Sep 28, 2011

Can i get the id in the result of serialize?... The serialize actually ignored the PK

Comment by anshul.s...@apra-infotech.com, Nov 4, 2011

my models.py is :

class Projects(models.Model):

projectName =models.CharField?(max_length = 100,unique=True,db_index=True) projectManager = EmbeddedModelField?('Users')

class Teams(models.Model):

teamType = models.CharField?(max_length =100) teamLeader = EmbeddedModelField?('Users') teamProject = EmbeddedModelField?('Projects')

class Users(models.Model):

name = models.CharField?(max_length = 100,unique=True) designation = models.CharField?(max_length =100 ) teams = ListField?(EmbeddedModelField?('Teams'))

when i do:

print serializers.serialize('json', Projects.objects.all(), indent=4,relations=('Users'))

It outputs:

[
{
"pk": "4eb3b7d0e814520db4000000", "model": "RESTAPI.projects", "fields": {
"projectName": "HELLO", "projectManager": "Users object"
}
}
]

It doesnot convert the Users object into Json.........Please help(I am using Mongodb as my database


Sign in to add a comment
Powered by Google Project Hosting