My favorites | Sign in
Project Home Downloads Issues Source
New issue   Search
for
  Advanced search   Search tips   Subscriptions
Issue 45: Serializing/Deserializing simple maps should be better supported
5 people starred this issue and may be notified of changes. Back to list
Status:  Fixed
Owner:  joel.leitch@gmail.com
Closed:  Sep 2008


Sign in to add a comment
 
Reported by azeckoski, Sep 15, 2008
Attempting to simply serialize a map results in fairly useless output:

// dead simple map
Map<String, Object> m = new ArrayOrderedMap<String, Object>();
m.put("id", 123);
m.put("thing", "AZ");

// serialize (annoyingly requires the typetoken thing)
String encoded = gson.toJson(data, new TypeToken<Map<String, Object>>()
{}.getType());

encoded = {"id":{},"thing":{}}

It should have been:
encoded = {"id":123,"thing":"AZ"}

This should really be able to handle the simple example of a map of
primitive/simple objects much better. The same type of thing happens when
attempting to read back in the data.

With a more realistic example it is even worse because another map placed
inside the first map results in an exception like so:
com.google.gson.JsonParseException: The JsonSerializer
com.google.gson.DefaultTypeAdapters$MapTypeAdapter@b27bb5 failed to
serialized object {name=aaron, date=Mon Sep 15 11:58:33 BST 2008, num=456,
array=[Ljava.lang.String;@fe3238} given the type class java.lang.Object
	at
com.google.gson.JsonSerializerExceptionWrapper.serialize(JsonSerializerExceptionWrapper.java:61)
	at
com.google.gson.JsonSerializationVisitor.visitUsingCustomHandler(JsonSerializationVisitor.java:177)
	at com.google.gson.ObjectNavigator.accept(ObjectNavigator.java:144)
	at
com.google.gson.JsonSerializationContextDefault.serialize(JsonSerializationContextDefault.java:47)
	at
com.google.gson.DefaultTypeAdapters$MapTypeAdapter.serialize(DefaultTypeAdapters.java:301)
	at
com.google.gson.DefaultTypeAdapters$MapTypeAdapter.serialize(DefaultTypeAdapters.java:293)
	at
com.google.gson.JsonSerializerExceptionWrapper.serialize(JsonSerializerExceptionWrapper.java:48)
	at
com.google.gson.JsonSerializationVisitor.visitUsingCustomHandler(JsonSerializationVisitor.java:177)
	at com.google.gson.ObjectNavigator.accept(ObjectNavigator.java:144)
	at
com.google.gson.JsonSerializationContextDefault.serialize(JsonSerializationContextDefault.java:47)
	at com.google.gson.Gson.toJson(Gson.java:272)
	at com.google.gson.Gson.toJson(Gson.java:228)
	at
org.sakaiproject.entitybroker.impl.EntityEncodingManager.encodeData(EntityEncodingManager.java:586)
	at
org.sakaiproject.entitybroker.impl.EntityEncodingManagerTest.testEncode(EntityEncodingManagerTest.java:243)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:585)
	at junit.framework.TestCase.runTest(TestCase.java:154)
	at junit.framework.TestCase.runBare(TestCase.java:127)
	at junit.framework.TestResult$1.protect(TestResult.java:106)
	at junit.framework.TestResult.runProtected(TestResult.java:124)
	at junit.framework.TestResult.run(TestResult.java:109)
	at junit.framework.TestCase.run(TestCase.java:118)
	at junit.framework.TestSuite.runTest(TestSuite.java:208)
	at junit.framework.TestSuite.run(TestSuite.java:203)
	at
org.eclipse.jdt.internal.junit.runner.junit3.JUnit3TestReference.run(JUnit3TestReference.java:130)
	at
org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
	at
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:460)
	at
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:673)
	at
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:386)
	at
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:196)
Caused by: java.lang.IllegalArgumentException: Map objects need to be
parameterized unless you use a custom serializer. Use the
com.google.gson.reflect.TypeToken to extract the ParameterizedType.
	at com.google.gson.TypeInfoMap.<init>(TypeInfoMap.java:34)
	at
com.google.gson.DefaultTypeAdapters$MapTypeAdapter.serialize(DefaultTypeAdapters.java:298)
	at
com.google.gson.DefaultTypeAdapters$MapTypeAdapter.serialize(DefaultTypeAdapters.java:293)
	at
com.google.gson.JsonSerializerExceptionWrapper.serialize(JsonSerializerExceptionWrapper.java:48)
	... 31 more



Sep 15, 2008
#1 azeckoski
Extra info on the deserializing:
json = {"id":123,"thing":"AZ"}

Map<String, Object> decoded = gson.fromJson(data, new TypeToken<Map<String,
Object>>() {}.getType());

produces a map with: {id=java.lang.Object@e6612c, thing=java.lang.Object@d704f0}
(seems to be instances of Object with no data)

Sep 16, 2008
Project Member #2 joel.leitch@gmail.com
First off, I'd like to start with some background information.  When you are defining
types (or local variables) that have type parameters, the JVM drops the actual type
parameters and associates everything as "Object".  This is known as "type erasure". 
In order for a Java Program to retrieve the actual type parameters at run-time, you
need to leverage the TypeToken object (this methodology was established by GUICE ---
see
http://google-guice.googlecode.com/svn/trunk/javadoc/com/google/inject/TypeLiteral.html).

Originally, we implemented Gson so that it could "serialize" these kinds of objects
without requiring the use of TypeToken; however, deserializing it back into the real
object requires it since the JSON output has no type information in it.  As well,
with this approach, it meant you were serializing the "real" object which meant that
some fields on the real object would be added to the JSON output.  Therefore, if you
had a List<A> and added both A and B objects (i.e. B extends A) some objects would
expose extra fields in the output.  We decided to take the more explicit route and
force the client to provide us the type parameters of the top-level object that is
being serialized.

The common use of a Map or List is that you populate the List with the same object
types.  Passing in Object means that you can add any instance of a class that you
desire to the data structure.  I know there are exceptions to this best practice, but
we do not want to implement this corner case scenario.  Instead, if you really do
want to use a list of Objects, then do as the exception message says and write a
"custom" (de)serializer (you can bind it specifically to a Map<String, Object> and
have the default Gson map serializer handle everything else).

As for a Map of Maps (i.e. Map<String, Map<String, Integer>>) this is already
supported and works well as long as you pass in the actual type object (i.e. new
TypeToken<Map<String, Map<String, Integer>>>() {}.getType())

Here's an example:
public static void main(String[] args) {
  Type mapType = new TypeToken<Map<String, Map<String, Integer>>>() {}.getType();
  Map<String, Map<String, Integer>> map = new HashMap<String, Map<String, Integer>>();
  Map<String, Integer> value1 = new HashMap<String, Integer>();
  value1.put("lalala", 78);
  value1.put("haha", 9999);
  map.put("id", value1);
    
  Map<String, Integer> value2 = new HashMap<String, Integer>();
  value2.put("nahhd", 121112);
  value2.put("uuywss", 19987);
  map.put("thing", value2);
    
  Map<String, Integer> value3 = new HashMap<String, Integer>();
  map.put("other", value3);

  Gson gson = new Gson();
  String json = gson.toJson(map, mapType);
  System.out.println(json);
    
  Map<String, Map<String, Integer>> deserializedMap = gson.fromJson(json, mapType);
  System.out.println(deserializedMap);
}

=========== OUTPUT ===========
{"thing":456,"id":123}
{"other":{},"thing":{"nahhd":121112,"uuywss":19987},"id":{"lalala":78,"haha":9999}}
{other={}, thing={nahhd=121112, uuywss=19987}, id={lalala=78, haha=9999}}


For now, I am closing this off as "Working as Designed".  Maybe I am not completely
following your issue and if that is the case, please start up a new discussion in our
Gson discussion group.

Thanks,
Joel
Status: Invalid
Owner: joel.leitch
Sep 18, 2008
#3 azeckoski
We solved our problem by using a different library but I wanted to put a comment here
anyway.

So what happens if I want to do this?
Map<String, Number>
or
Map<String, Serializable>

(it seems to fail)

It seems that this is designed to only work for the basic case where I have really
simple and non-nested structures where all the beans are easily instantiable and not
superclasses. It is a shame that this is considered working as designed.

Sep 27, 2008
Project Member #4 joel.leitch@gmail.com
I am glad to hear that you found something that works for you, but it's too bad you
are unable to use Gson.  I'd still like to follow up on this issue because it is
user's like yourself that will help to advance this library.

First off, are you serializing and deserializing an object of type Map<String,
Number>?  If it is serialization only, than that is a much "easier" problem to solve
because we have the runtime types.  As for "deserializing" this kind of object, we
have provided our clients with the concept of a custom "Type Adapter".  You should be
able to write a type as follows to get it to work with "Number":

  public static class NumberTypeAdapter 
      implements JsonSerializer<Number>, JsonDeserializer<Number>,
InstanceCreator<Number> {

    public JsonElement serialize(Number src, Type typeOfSrc, JsonSerializationContext
context) {
      return new JsonPrimitive(src);
    }

    public Number deserialize(JsonElement json, Type typeOfT,
JsonDeserializationContext context)
        throws JsonParseException {
      JsonPrimitive jsonPrimitive = json.getAsJsonPrimitive();
      if (jsonPrimitive.isNumber()) {
        return jsonPrimitive.getAsNumber();
      } else {
        throw new IllegalStateException("Expected a number field, but was " + json);
      }
    }

    public Number createInstance(Type type) {
      return 1L;
    }
  }

  public static void main(String[] args) {
    Map<String, Number> map = new HashMap<String, Number>();    
    map.put("int", 123);
    map.put("long", 1234567890123456789L);
    map.put("double", 1234.5678D);
    map.put("float", 1.2345F);
    Type mapType = new TypeToken<Map<String, Number>>() {}.getType();

    Gson gson = new GsonBuilder().registerTypeAdapter(Number.class, new
NumberTypeAdapter()).create();
    String json = gson.toJson(map, mapType);
    System.out.println(json);
    
    Map<String, Number> deserializedMap = gson.fromJson(json, mapType);
    System.out.println(deserializedMap);
  }

========== OUTPUT ==========
{"double":1234.5678,"float":1.2345,"int":123,"long":1234567890123456789}
{double=1234.5678, float=1.2345, int=123, long=1234567890123456789}


We should probably just include the above type adapter as a default in Gson and I
will discuss this with Inderjeet.  There is a bug, however, since you actually have
to specify a "instance creator" for this type of object (i.e. primitive), but I will
have that fixed by the next release.  You should be able to write something similar
as above for "Serializable".

I hope this information is helpful and thanks for the all the feedback on this library.
Oct 21, 2008
#5 zhaojinz...@gmail.com
I modify the MapTypeAdapter to match the jdk14's Map
======================================================
public JsonElement serialize(Map src, Type typeOfSrc, JsonSerializationContext context) {
      JsonObject map = new JsonObject();
      //Type childType = new TypeInfoMap(typeOfSrc).getValueType();
      for (Iterator iterator = src.entrySet().iterator(); iterator.hasNext(); ) {
        Map.Entry entry = (Map.Entry) iterator.next();
		Object obj = entry.getValue();
        JsonElement valueElement = context.serialize(obj, obj.getClass());
        ---------------------------------------------------------------------modified
        map.add(entry.getKey().toString(), valueElement);
      }
      return map;
    }
--------------------------------------------------------------------------------

and then the map class can be used like this:
  HashMap aaaa = new HashMap();
  aaaa.put("aa", 1212);
  aaaa.put("bb", "fasdfa");
  System.out.println(gson.toJson(aaaa));
==========output=================
{"bb":"fasdfa","aa":1212}
=================================

It can run, good or bad? because there are lot's of  jdk14's source code in many
project's.

Oct 21, 2008
Project Member #6 inder123
Thanks for providing the code snippet. This will not work properly in case of 
genericized maps since in those cases it is important to use the type specified in 
the field declaration instead of the actual type. I have made similar fixes for  Issue 
54  and 58 that I will apply in this case as well. 
Oct 21, 2008
Project Member #7 inder123
I have fixed this issue in r277 

Now, you should be able to serialize raw maps. The deserialization continues to 
require parameterized type. 
Status: Fixed
Jan 22, 2010
#8 SystemIn...@gmail.com
Thank you for fixing this issue: 
In java land, you really shouldn't be instantiating Map<String, Object> but since
we're dealing with JSON world, it actually makes a lot of sense. 
Consider Map<String, Object> map;
map.put("field1", 123);
map.put("field2", "myfield2");

What is gson.toJson(map)???
It's a javascript object o where o.field1 is the number 123 and o.field2 is the
string myfield2!
Mar 5, 2010
#9 demog...@gmail.com
json convert to map<Integer,MyClass> it have problem !
how to  do ?

Jun 25, 2010
#10 mani.dor...@gtempaccount.com
If you do json eval() in javascript or python, you get a dictionary. Inside the dictionary, it has String/Number or nested dictionaries. eval doesnt expect these type declarations.
I would expect the same on static language as well - Maps with default Number (lossless datatype like Double) datatype for deserialization.
Aug 6, 2010
#11 kstruil...@gmail.com
Hi demograp,

Did you forget to implement a default constructor for MyClass ?
It was my case, and I solved it doing this.

Hope it helps.
Sep 1, 2011
#12 dska...@gmail.com
I faced similar problems. Easiest solution for me was wrapping the desired map in a wrapper object and passing that to gson. I guess in the end that just boils down to be the same as providing the TypeToken, but it is a much more straightforward solution for those who want a quick fix.
Sign in to add a comment

Powered by Google Project Hosting