My favorites | Sign in
Project Home Downloads Wiki Issues Source
Search
for
ParsingJSON  
Shows how to parse JSON datasets with svenson
Featured
Updated Oct 19, 2009 by ff...@gmx.de

Parsing JSON with svenson

Svenson tries to give you a maximum of flexibility when parsing JSON. From parsing JSON it a nested graph of maps and lists, to parsing it into a Object graph of your own classes with dynamic parts, every combination short of transforming the JSON prior to parsing it is possible.

Simple Parsing

Per default, JSON will just produce a nested graph of maps and lists. Here is an example of default parsing:

import org.svenson.JSONParser;

...
    // assume json to be a JSON datasets as String
    Object o = JSONParser.defaultJSONParser().parse(json);

o will now contain either a map or a list instance, depending on whether the first symbol in the JSON is '{' or '['.

Parsing into concrete types

While this can be useful if you have absolutely no idea about the JSON structure you are receiving, you usually do have an idea about that and so might want to define a root type:

import org.svenson.JSONParser;

...
    // assume json, json2 and json3 to be a JSON datasets as String
    Map m = JSONParser.defaultJSONParser().parse(Map.class, json);
   
    List m = JSONParser.defaultJSONParser().parse(List.class, json2);

    MyBean bean = JSONParser.defaultJSONParser().parse(MyBean.class, json3);

These are examples of parsing with a defined root type. JSONParser will try to parse the json dataset into the given type and throws an org.svenson.JSONParseException when that is not possible.

By default, arrays will be parsed into java.util.List instances and objects into java.util.Map instances. If you define beans as root type or as a type hint, objects can also be parsed into beans of that type, provided that all JSON properties have a corresponding java properties (or the bean implements DynamicProperties).

Apart from property access, there is another way to parse arrays into a bean property. A JSON dataset like

{
    "qux" : [ ... ],
}

can also be parsed into a class like

    public class Baz
    {
        ...
        public void addQux(Qux qux)
        {
            ...
        }
    }

where there is a method starting with "add" and ending on the same name as the corresponding JSON property.

@JSONProperty

See Generating JSON for how to influence the handling with the @JSONProperty annotation.

Interface Mapping

As you can see in the example above, you can use interfaces as root types. That is of course only possible if JSONParser knows, what concrete implementations of these interface it is supposed to generate. You can feed a map mapping an interface type to an implementing type to JSONParser#setInterfaceMappings(Map<Class,Class>)

Type Hints

We know now how to define a root type. But what if you want to define concrete types not at the root of the JSON structure, but deep within it?

This is where Type Hints come into play. With type hints you can define a concrete type to be used at a specific point within the JSON structure.

import org.svenson.JSONParser;

...
    // assume json to be an JSON dataset as String
    JSONParser parser = new JSONParser();
    parser.addTypeHint(".foos[]", Foo.class);

    Map result = parser.parse(Map.class, json);

Here you can see how to set up parser with one type hint that is able to parse JSON datasets like

    {
        "foos" : [ ... ]
    }

so that result is a Map with a key "foos" mapping to a list of Foo instances. For this to work, Foo needs to have a default constructor and the objects in the "foos" array must be parsable into Foo instances.

@JSONTypeHint

You can achieve the same effect by annotating a property method that has a list or map as property type with the org.svenson.JSONTypeHint.

import java.util.List;
import org.svenson.JSONTypeHint;

public class Foo
{
    private List<Bar> bars;

    @JSONTypeHint(Bar.class)
    public void setBars(List<Bar> bars)
    {
        this.bars = bars;
    }
}

This will ensure that the objects in the bar property array will be parsed into Bar instances.

{
    "bars" : [ ... ]
}

This is necessary because of the type erasure.

Dynamic Properties

If your classes need the ability to cope with abritrary properties whose names are not known at compile time, you can implement org.svenson.DynamicProperties or extend org.svenson.AbstractDynamicProperties

import org.svenson.AbstractDynamicProperties;

public class MyBean extends AbstractDynamicProperties
{
   private String foo;

   public String getFoo()
   {
       return foo;
   }

   public void setFoo(String foo)
   {
       this.foo = foo;
   }
}

Implemented this way, MyBean can both have regular java properties and can also be provided with dynamic properties with arbritrary names and values. These dynamic Properties can be accessed with the methods defined in the DynamicProperties interface.

Regular java properties will have precedence over dynamic ones, so in this example, the foo property will always be available via getFoo() and not via getProperty("foo")

...
public interface DynamicProperties
{
    /**
     * Sets the attribute with the given name to the given value.
     *
     * @param name
     * @param value if <code>null</code>, the attribute is removed.
     */
    void setProperty(String name, Object value);

    /**
     * returns value of the attribute with the given name.
     *
     * @param name
     */
    Object getProperty(String name);

    /**
     * Returns the set of available dynamic attribute names.
     *
     * @return
     */
    Set<String> propertyNames();
}

Type Mapper

Type Mappers are for now the most complex mechanism with which you can control the types used to parse your JSON messages. Classes implementing the TypeMapper interface can be registered by calling JSONParser#setTypeMapper(TypeMapper) ( If you really need to define more than one type mapper, use org.svenson.CompositeTypeMapper ).

...
/**
 * Inspects the current position of a tokenizer and decides what type to use at that position.
 *
 * @author shelmberger
 */
public interface TypeMapper
{
    /**
     * Returns the type to use for the current tokenizer position or <code>null</code>, if the type mapper does not want to
     * change the type used at this position. The type mapper must reset the tokenizer to a token stream position that allows
     * the parsing to continue. this will usually be the tokenizer position on method entry. (i.e. the first token received by the
     * type mapper needs to be fed to {@link JSONTokenizer#pushBack(org.svenson.tokenize.Token)}
     *
     * @param tokenizer         tokenizer to get the tokens from. needs to set to a position behind the inspected value to continue
     *                          parsing correctly.
     *
     * @param parsePathInfo     the current parsing path within the root object
     * @param typeHint          initial type hint as configured by {@link JSONParser#addTypeHint(String, Class)}, can also be <code>null</code>.
     *
     * @return type hint or <code>null</code>, if the type mapper does not want to
     * change the type used at this position.
     */
    Class getTypeHint(JSONTokenizer tokenizer, String parsePathInfo, Class typeHint);
}

They can inspect the token stream generated by the provided JSONTokenizer and decide to overide the configured type hint.

PropertyValueBasedTypeMapper

org.svenson.PropertyValueBasedTypeMapper is one predefined TypeMapper that comes with the svenson distribution. It allows you to select the type to parse a JSON object into based on the value of a property of that JSON object.

For example a data-structure like

{
    "total_rows": 3,
    "offset": 0,
    "rows": [{ "type":"foo", "value":"aaa" },{ "type":"bar", "value":"bbb" },{ "value":"ccc","type":"bar"  }]
}

can be parsed with the following code:

        JSONParser parser = new JSONParser();
        PropertyValueBasedTypeMapper mapper = new PropertyValueBasedTypeMapper();
        mapper.setParsePathInfo(".rows[]");
        mapper.addFieldValueMapping("foo", Foo.class);
        mapper.addFieldValueMapping("bar", Bar.class);
        parser.setTypeMapper(mapper);

The type mapper will ensure that the values of the rows array will be mapped to Foo and Bar instances based on the value of they "type" property.

ClassNameBasedTypeMapper

The org.svenson.ClassNameBasedTypeMapper is another type mapper delivered with svenson that works like the PropertyValueBasedTypeMapper but does not require type configuration but instead uses either fully or partially qualified class names with optional base type enforcement.

Parsing from a input stream

Since svenson 1.3.0 you can also parse JSON directly from an input stream.

import java.io.FileInputStream;
import java.io.IOException;
import java.util.Map;

import org.svenson.JSONParser;
import org.svenson.tokenize.InputStreamSource;

...
        // assume inputStream to be an input stream delivering JSON data
        JSONParser parser = JSONParser.defaultJSONParser();        
        InputStreamSource source = new InputStreamSource( inputStream, true);
        Map m = parser.parse(Map.class, source);

Object Factories

You can now also create object factories by implementing the org.svenson.ObjectFactory interface

public interface ObjectFactory<T>
{
    /**
     * Returns <code>true</code> if the factory can create objects of the given class
     * 
     * @param cls
     * @return
     */
    boolean supports(Class<T> cls);
    
    /**
     * Creates an instance of the given type.
     * @param typeHint
     * @return
     */
    T create(Class<T> typeHint);
}

and registering your instance by calling JSONParser#addObjectFactory(org.svenson.ObjectFactory).

The future

I might add even more flexibility when I see use cases for it that are general enough to be included. Object factories, JSON transformation.. A lot would be possible but for now I have no clear vision of that.

So if your requirements are not met by svenson, you could try contacting me to give me a clearer idea of what could be added.

Comment by jma...@translab.its.uci.edu, Jul 28, 2009

There is a typo in the example for JSONTypeHint. There should not be a semicolon after @JSONTypeHint(Bar.class) It should be:

@JSONTypeHint(Bar.class)
public void setBars(List<Bar> bars)
{
    this.bars = bars;
}
Comment by project member ff...@gmx.de, Oct 19, 2009

@jmarca

Thanks for catching this.

The property also must be "bars" and not "bar". No funky plural recognition ;)

Comment by joscha.f...@gmail.com, Dec 29, 2009

How can I use the PropertyValueBasedTypeMapper? in conjunction with a response like:

[{},{},{}]

where is the root element - setting the parse path info to "." did not seem to work...

Comment by project member ff...@gmx.de, Dec 30, 2009

@Joscha

The parse path info for to convert the three objects in

[{},{},{}]

must be "[]" .

The components are either ".something" for a "something" property or "" for an array. If you need more complicated matching, you have to use a PathMatcher.

Comment by eick...@gmail.com, Sep 12, 2010

There seems to be a difference between using addTypeHint and @JSONTypeHint (a bug?). When I use @JSONTypeHint, the nested class can only contain String primitives without further type hints (otherwise I get exceptions). With addTypeHint, the nested class can contain any primitive type without further type hints. I debugged into 1.3.5 and 1.3.8 and it seems that "memberType" in ParseContext? is screwed up when using the annotation.

Cheers, André

Comment by eick...@gmail.com, Sep 12, 2010

P.S.: Can I parse into nested classes?

Comment by project member ff...@gmx.de, Sep 12, 2010

@andre

There are test cases for nested bean scenarios. You only need @JSONTypeHint to provide the information about the type erased by type erasure. Nested Beans should just work on any level.

Hard to say anything more without details.

Comment by eick...@gmail.com, Sep 13, 2010

That's right. Below is a sketch of what I mean. This code does not work as it is. To make it work you have to move out the nested classes into top-level classes. The second thing is that you have to replace the @JSONTypeHint annotation by the addTypeHint method. (Actually, this was supposed to deal with Couch changes API, but then I discovered the implementation in jcouchdb :-) )

public class TestNesting {
	public static final String JSON = "{\"seq\":1,\"id\":\"067911f608c9b19fc404c9930fbdcfd0\",\"changes\":[{\"rev\":\"1-acf2a9444aa0064249190548103cc0b4\"}],\"doc\":{\"_id\":\"067911f608c9b19fc404c9930fbdcfd0\",\"_rev\":\"1-acf2a9444aa0064249190548103cc0b4\",\"ALARM_TYPE\":4,\"PROBABLE_CAUSE\":1017,\"CONSEC_NBR\":2642,\"ALARM_TIME\":1271875656000.0,\"CORRELATED_ALARM\":0,\"CANCEL_TIME\":\"\",\"CORRELATING_ALARM\":0,\"INTERNAL_ALARM\":0,\"ADDITIONAL_INFO_6\":\"\",\"ADDITIONAL_INFO_5\":\"\",\"ACK_STATUS\":1,\"ADDITIONAL_INFO_7\":\"\",\"ADDITIONAL_INFO_2\":\"\",\"ADDITIONAL_INFO_1\":\"Motion detected in zone 1\",\"ADDITIONAL_INFO_4\":\"\",\"ADDITIONAL_INFO_3\":\"\",\"ORIGINAL_SEVERITY\":99,\"ALARM_ID\":25,\"CORRELATOR_CREATED\":0,\"ACK_TIME\":\"\",\"PERCEIVED_SEVERITY\":3,\"FILTERED_FROM\":0,\"ALARM_DN\":\"Network-USA/Network-CA/GW-SantaClara/Site-1/Camera-8.0\",\"ALARM_TEXT\":\"MOTION DETECTED\",\"INSERT_TIME\":1271875659000.0,\"NE_AGENT_MR_GID\":0,\"NE_GID\":3000000002380.0,\"ALARM_STATUS\":1,\"SPECIFIC_PROBLEM\":100002,\"NE_MR_GID\":0,\"AGENT_GID\":3000000000750.0,\"CANCELLED_BY\":\"\",\"UPDATE_TIMESTAMP\":1271900859390.0,\"ACKED_BY\":\"\",\"NE_SITE_GID\":0}}";
	
	@Test
	public void testNesting()
	{
		JSONParser parser = JSONParser.defaultJSONParser();
		//parser.addTypeHint(".doc", FmAlarm.class);
		parser.parse(EventNotification.class, JSON);
	}
	
	public class FmAlarm extends AbstractDynamicProperties {
		@JSONProperty(value = "ALARM_TYPE", ignoreIfNull = true)
		public int getAlarmType() {
			return this.alarmType;
		}

		public void setAlarmType(int alarmType) {
			this.alarmType = alarmType;
		}

		private int alarmType;
	}

	public class EventNotification extends AbstractDynamicProperties {
		public FmAlarm getDoc()
		{
			return doc;
		}

		@JSONTypeHint(FmAlarm.class)
		public void setDoc(FmAlarm doc)
		{
			this.doc = doc;
		}
		
		private FmAlarm doc;		
	}	
}
Comment by project member ff...@gmx.de, Sep 13, 2010

The main problem with your inner classes is that they're not defined as static, thus they need an TestNesting instance to be created.

In Java this would be something horrible like

        new TestNesting().new EventNotification();

When invoked from a static context.

Then there is a second bug apparently triggered by the superfluous @JSONTypeHint which seems indeed to confuse the type system. Will surely investigate that, but since it's not needed in this case, you are fine just by adding a "static" to your inner class definitions and removing the @JSONTypeHint.

Comment by eick...@gmail.com, Sep 13, 2010

Oh right ... thanks for the comment!

Cheers, André

Comment by jco...@gmail.com, Nov 1, 2010

How would I parse into a generic root type as in the following example using GSON? Also, I see how I could use a type hint to get A#listOfBs to parse correctly, but how do I do the same for Maps?

public class A {

    public List<B> listOfBs;

    public Map<C, D> mapOfCsToDs;

}

Gson gson = new Gson();
List<A> listOfAs = gson.fromJson(listOfAsJson, new TypeToken<List<A>>(){}.getType());
Map<String, A> mapOfStringsToAs = gson.fromJson(mapOfStringsToAs, new TypeToken<Map<String, A>>(){}.getType());
Comment by project member ff...@gmx.de, Nov 2, 2010

@jcottr:

Since the C in your example has to be always String for JSON, this reduces to

import org.svenson.matcher.RegExPathMatcher;
import org.svenson.JSONParser;
…
JSONParser jsonParser = new JSONParser();
jsonParser.addTypeHint(new RegExPathMatcher("\\..*"), D.class);
Map<String,D> someThings = jsonParser.parse(Map.class, json);

for a simple map.

Comment by samuelg...@gmail.com, Nov 13, 2010

Parsing json frmo a InputStream? doesn't seem to be available on the svn repository:

http://code.google.com/p/svenson/source/browse/trunk/src/org/svenson/JSONParser.java

Comment by project member ff...@gmx.de, Nov 15, 2010

You just need to wrap the input stream in a org.svenson.tokenize.InputStreamSource? .

Comment by sameer...@gmail.com, Jan 13, 2011

Is it possible to enforce the properties to be required in the input JSON? something like @JSONProperty(required=true) which will throw an exception in case the property is missing.

Comment by project member ff...@gmx.de, Jan 14, 2011

So far, there is no support for validation / required properties whatsoever.

Comment by marcin.c...@gmail.com, Mar 24, 2011

Hi

I'm trying to use jcouchdb (https://code.google.com/p/jcouchdb/) for accessing my CouchDB instance from Java. I have some JSon documents that I'd like to parse into Java classes - with Svenson, used in jcouchdb, and then put those parsed objects into DB. I generate this JSON objects with AVRO (http://avro.apache.org) JSon Encoder, they seem to be ok, but apparently other parsers have problems with them.

My JSon strings look like this:

{
   "id":40,
   "event_id":"48764322212",
   "note":{
      "string":"ABC note"
   },
   "created_date":null,
   "event_category":null,
   "city":null,
   "address":null
}

Which seems valid JSON - validated with http://jsonformatter.curiousconcept.com/

However my Svenson object defined like this:

public class Note {

    Long id;
    String eventId;
    String note;
    String createdDate;
    String eventCategory;
    String city;
    String address;

    @JSONProperty()
    public Long getId() {

    @JSONProperty("event_id")
    public String getEventId() {

    @JSONProperty("note")
    public String getNote() {

    @JSONProperty("created_date")
    public String getCreatedDate() {

    @JSONProperty("event_category")
    public String getEventCategory() {

    @JSONProperty("city")
    public String getCity() {

    @JSONProperty("address")
    public String getAddress() {

}

(setters and getters' bodies intentionally removed)

The error when parsing is:

Cannot set property string on class java.lang.String

It seems that this JSON is parsed correctly (there is a difference in note field):

{
   "id":40,
   "event_case_id":"000-123123123",
   "event_msisdn":"48764322212",
   "note":"Planowana data portacji: 2011/01/27 11:42:49",
   "created_date":null,
   "event_category":null,
   "city":null,
   "address":null
}

How can I work this out? Perhaps there is another json library that would work for me?

Comment by project member ff...@gmx.de, Mar 24, 2011

@marcin:

Svenson does not really support structural transformation of JSON apart from using custom type converters.

In you case, you have a note string property in your Note class which does not really fit the complex object in your JSON

The most easy solution would be to define a NoteContent? class like this

public class NoteContent
{
    private String string;

    public String getString()
    {
        return string;
    }

    public void setString(String string)
    {
        this.string = string;
    }
}

and changing the note property in Note to be of the Type NoteContent?.

A type converter for the NoteContent? structure would look somewhat like http://svenson.googlecode.com/svn/trunk/test/org/svenson/converter/ComplexDateConverter.java

Comment by lati2...@gmail.com, Jun 3, 2011

How to read json's object value which is keyed by "@key-name?"

for example:

[{"diskVolumePair":{"@id":"101",

"compressionEnabled":false, "link":[{"@title":"Source_101",
"@href":"abc", "@rel":"sourceDiskVolume"},
{"@title":"Dest_101",
"@href":"def", "@rel":"destinationDiskVolume"}],
"name":"Exchange_101", "online":false}}]
It is a valid json.

How should I access the value against id/ title/ href and rel keys, since they have '@' appended in the beginning?

Comment by codec...@gmx.net, May 12, 2012

Hi, I have a problem getting this to work...

given: - 2 types ActionList? and Action - ActionList? can have multiple dynamic properties of type Action

setter code: actionList.setProperty("foo", new Action()); actionList.setProperty("bar", new Action());

getter code: Object value = actionList.getProperty("foo")

expected: value instanceof Action

what happens: value is instance of HashMap?

I was able to work around this by setting a TypeHint? with PathMatcher?, but this kind of mapping is global for parser instance and not bound to a specific type (e.g. ActionList?)

Do is miss something? Do you have a tip?

regards Stefan


Sign in to add a comment
Powered by Google Project Hosting