|
PerformanceTesting
Performance tesing results
IntroductionJSON.simple uses json-test-suite to do performance testing. This is a simple performance testing against the reference implementation of JSON.org in a very simple scenario. It does not cover all aspects of applications in the real world. Its purpose is to provide a reference point to the user. Steps
ant -f test.xml perf TEST-org.json.test.perf.SimpleTest.txt ResultsThe following is an arbitrarily selected result of ones generated by the above command. The millisecond value after each parser is measured by the time needed by parsing and retrieving the desired value from the sample data. The length of the sample data is about 620K characters (more precisely, UTF-16 code units) and the value '6.908319653520691E8' is a randomly generated value used for validating of the parsing result. ==sample== len:621807 org.json.JSONObject :: 103.1 ms 6.908319653520691E8 org.json.simple.JSONParser :: 14.1 ms 6.908319653520691E8 org.json.simple.JSONParser with ContentHandler :: 5.5 ms 6.908319653520691E8 How does it work?Please refer SimpleTest.java for details. You can also test JSON.simple against other libraries by removing the comment tags of SimpleTest.java. You may need to put their jar files and dependencies in directory 'lib' before running the command above. |
► Sign in to add a comment
11
Looking at SimpleTest?.java, this performance test just finds the value of a single element in a largish input JSON. The implementation with org.json reads and stores the entire JSON contents (in a Java Map), and then retrieves one of the entries. The implementation with org.json.simple.JSONParser with ContentHandler? reads and does not store the JSON contents until the target element is found.
So, this isn't really an apples-to-apples comparison of the performance of the two libraries. It's just demonstrating that JSON.simple doesn't have to read and store the entire JSON contents just to pick out one small part. Also, this performance test doesn't match the vast majority of real-world JSON parsing concerns, in my experience, as it's rare that folks want just one small value from a large JSON input, and it's much more common that folks want to read and store (somehow) all or most of the input.
I recently got involved in providing serializer/deserializer implementations for the jvm-serializers project, as I wanted to know how different Java-to/from-JSON libraries actually performed, but half of the projects all claim that their implementation is fast. The most recent test results are available at https://github.com/eishay/jvm-serializers/wiki/ .
I wrote one implementation using JSON.simple's manual/tree processing API, and it showed to be slightly slower than org.json overall. Also, I just submitted another serializer/deserializer using a custom JSON.simple ContentHandler?, finely tuned to solve the specific problem this test suite presents. While it showed to be around 10% faster at deserializing the contents compared to JSON.simple or org.json with manual/tree processing, it's nothing close to the claim I thought was made looking at this page that JSON.simple is almost 20x faster than org.json.
I welcome any suggestions for further improvements of the JSON.simple serializers/deserializers implementations at https://github.com/eishay/jvm-serializers. They're in the tpc/src/serializers/ directory. (Also, I'm glad to provide a simplified version of this project, if anyone wants to help improve the implementation, but doesn't want to figure out navigating, compiling, and running the entire project.)