My favorites | Sign in
Project Home Downloads Wiki Issues Source
New issue   Search
for
  Advanced search   Search tips   Subscriptions
Issue 501: Refactor XML processing to enhance memory usage
1 person starred this issue and may be notified of changes. Back to list
Status:  Done
Owner:  janne.t....@gmail.com
Closed:  Feb 2011
Cc:  pekka.klarck


Sign in to add a comment
 
Project Member Reported by pekka.klarck, Mar 14, 2010
Because RF 2.5 has Python/Jython 2.5 as the minimum requirement, we can
always use ElementTree for parsing the XML outputs. Having just one way to
process the outputs internally makes the code cleaner, but more importantly
it will be easier to enhance the memory usage. We probably can use the
`iterparse` method as illustrated in the latest episode of the excellent
PyMOTW series:
http://blog.doughellmann.com/2010/03/pymotw-parsing-xml-documents-with.html#watching-events-while-parsing

As part of this refactoring, we can/should probably remove the old
`utils.DomWrapper` altogether. That will affect some of the supporting
tools, but they can be updated to use ElememtTree instead. Because others
might use the `DomWrapper` too, this change is potentially backwards
incompatible. 
Mar 31, 2010
#1 c.martin...@gmail.com
Does this statement "Because RF 2.5 has Python/Jython 2.5 as the minimum requirement"
mean that in order to use RF 2.5 I will need BOTH Python and Jython installed?
Mar 31, 2010
Project Member #2 pekka.klarck
No, Python or Jython is enough. With both the version must be >= 2.5 but < 3.0.
Apr 14, 2010
Project Member #3 pekka.klarck
ElementTree can nowadays be used with all the supported Python/Jython versions and I
removed old javadomwrapper and minidomwrapper in r2972.
Apr 20, 2010
Project Member #4 KariH...@gmail.com
Descoped from 2.5.
Labels: -Target-2.5
Jan 31, 2011
Project Member #5 jpran...@gmail.com
(No comment was entered for this change.)
Labels: Target-2.5.6
Feb 1, 2011
Project Member #6 janne.t....@gmail.com
(No comment was entered for this change.)
Status: Started
Owner: janne.t.harkonen
Feb 2, 2011
Project Member #7 janne.t....@gmail.com
The readers module now uses elementtree directly. I made some not-so-scientific performance benchmarks, and seems that at least the memory consumption went down considerably (~40%). 

The code in readers is not the most beautiful, and I did not even try to clean it up considerably.


Status: Review
Cc: pekka.klarck
Feb 2, 2011
Project Member #8 pekka.klarck
Janne's changes in r4579 look good. Hopefully we can rewrite the whole xml reading and html generation logic in 2.6.
Status: Done
Feb 2, 2011
Project Member #9 pekka.klarck
Performance improvements with larger files were so big that it's fair to make this high priority enhancement.
Labels: -Type-Refactoring -Priority-Medium Type-Enhancement Priority-High
Sign in to add a comment

Powered by Google Project Hosting