It's definitely a new feature; it is only a problem when you use to_dict() and 
try to pass the result back in. That's not what to_dict() was originally meant 
for -- it was just meant as a helper to generate JSON. But given that we added 
this feature to non-Expando Models (as opposed to introducing a separate 
from_dict() class method) we should also add it to Expando.

Fantastic.  I think this takes the concept from issue 207 a step further by not 
only copying content between entities, but allowing nested structure to be 
dynamically copied too in the case of expandos!

Just for completeness the other case i can think of is models inside expandos 
or expandos inside models arbitrarily deep.  I dont know if this will be 
implicitly implemented by this issue, but I just thought I'd point it out.

Yes, mutual inclusion should work too. I'll make sure to include it explicitly 
in the unit test.

FWIW, I'm pretty busy -- do you feel up to suggeting a patch? It should be in 
the same place where the Expando code currently chooses between 
StructuredProperty and GenericProperty.

Sure I'll give this a shot and if i don't make much headway i'll pass it back 
to you.  I believe the section of code you are referring to is 
Expando.__setattr__:

* relevant excerpt from model.py *
    if isinstance(value, Model):
      prop = StructuredProperty(Model, name)
    else:
      repeated = isinstance(value, list)
      indexed = self._default_indexed
      # TODO: What if it's a list of Model instances?
      prop = GenericProperty(name, repeated=repeated, indexed=indexed)

Yup, that's it. Writing a test should be easy enough. :-)

Hi Guido,

I have a simple patch thanks to your hint:

* new excerpt from model.py *
    if isinstance(value, Model):
      prop = StructuredProperty(Model, name)
    elif isinstance(value, dict):                      <-- added line
      prop = StructuredProperty(Expando, name)         <-- added line
    else:
      repeated = isinstance(value, list)
      indexed = self._default_indexed
      # TODO: What if it's a list of Model instances?
      prop = GenericProperty(name, repeated=repeated, indexed=indexed)

This patch satisfies the case of transforming json with nested data into a 
nested set of expandos and thus passes the unit test from my original comment 
in this thread.  It does not appear to support mutual inclusion to arbitrarily 
deep levels however.  For that I propose another method: 
_populate_from_model(some_populated_expando_instance).  Unlike my patch, this 
method would have access to the actual expando model structure rather than just 
a dict of dicts and thus would know how to create and populate models at deeper 
levels. I'll leave this method for another feature request.

Below are applicable unit tests and their results.  You'll notice that the ones 
that fail only due so because the populated models use the base Expando class 
rather than the Expando subclasses of the originating models.  This is 
something that is probably satisfactory and could be resolved with the proposed 
_populate_from_model feature I believe.

    def test_model_in_model(self):
        """Passes"""
        class B(ndb.Model):
            b1 = ndb.StringProperty()
        class A(ndb.Model):
            a1 = ndb.StructuredProperty(B)
        a = A(a1 = B(b1 = 'b1test'))
        new_a = A(**a.to_dict())
        self.assertEqual(a, new_a)

    def test_expando_in_model(self):
        """Passes"""
        class E(ndb.Expando):
            pass
        class M(ndb.Model):
            m1 = ndb.StructuredProperty(E)
        a = M(m1 = E(e1 = 'e1test'))
        new_a = M(**a.to_dict())
        self.assertEqual(a, new_a)

    def test_expando_in_expando(self):
        """AssertionError: A(a1=B(b1='b1test')) != A(a1=Expando(b1='b1test'))"""
        class B(ndb.Expando):
            pass
        class A(ndb.Expando):
            pass
        a = A(a1 = B(b1 = 'b1test'))
        new_a = A(**a.to_dict())
        self.assertEqual(a, new_a)

    def test_model_in_expando(self):
        """AssertionError: E(e1=M(m1='e1test')) != E(e1=Expando(m1='e1test'))"""
        class E(ndb.Expando):
            pass
        class M(ndb.Model):
            m1 = ndb.StringProperty()
        a = E(e1 = M(m1 = 'e1test'))
        new_a = E(**a.to_dict())
        self.assertEqual(a, new_a)

    def test_expando_in_expando_without_structure(self):
        """AssertionError: A(a1=B(b1='b1test'), c1='c1test') != A(a1=Expando(b1='b1test'), c1='c1test')"""
        class A(ndb.Expando):
            pass
        class B(ndb.Expando):
            pass
        a = A(a1 = B(b1 = 'b1test'))
        a_new = A(c1 = 'c1test')
        a_new.populate(**a.to_dict())
        a.c1 = 'c1test'
        self.assertEqual(a, a_new)

    def test_expando_in_expando_with_structure(self):
        """AssertionError: A(a1=B(b1='b1test')) != A(a1=Expando(b1='b1test'))"""
        class A(ndb.Expando):
            pass
        class B(ndb.Expando):
            pass
        a = A(a1 = B(b1 = 'b1test'))
        a_new = A(a1 = B())
        a_new.populate(**a.to_dict())
        self.assertEqual(a, a_new)

    def test_expando_in_expando_with_lists(self):
        """Passes"""
        class B(ndb.Expando):
            pass
        class A(ndb.Expando):
            pass
        a = A(a1 = [B(b1 = [0,1,2,3]),B(b2='b2test')])
        new_a = A(**a.to_dict())
        self.assertEqual(a, new_a)

Let me know if you would like me to commit this to your codebase or how you 
would like to proceed.  Thanks!

Jesse

Hi Jesse! That patch looks good. If you could get a checkout of NDB from the 
code tab above and apply it there, then you can easily get a diff using hg 
diff; attach that here and I'll apply it. But if you feel that's beyond your 
current powers I'll gladly apply the changes from your description above. (OTOH 
if you plan to contribute more, you might want to practice on this patch and 
maybe even use codereview.appspot.com to send me a code review -- use the 
update.py tool that that website suggests you use when you click on Create 
Instance.)

In either case, please fill out Google's open source contributor form; see 
http://code.google.com/p/appengine-ndb-experiment/wiki/Contributing for 
instructions.

Ok I have submitted a patch and the contributor form.

http://codereview.appspot.com/6812043/

This issue was closed by revision dbc21706e27f.

Submitted. Thanks!

Hi Guido,

This will be released end of November?  Do you think there's anyway for my app 
engine app to run with this patch sooner (ie today) ?  Maybe uploading the ndb 
codebase as part of my application code?

You can check out the sandbox2 branch of ndb, copy that into your app, and 
change all your imports from "from google.appengine.ext import ndb" to "import 
ndb".

Awesome!  I think you win on fastest response time. :)

worked like a charm.  thanks again!

[deleted comment]

Great! Just be sure not to mix google.appengine.ext.ndb and ndb; they have 
separate event loops and their classes have different identities, so 
isinstance() checks in the code will fail.