My favorites | Sign in
Project Home Downloads Wiki Issues
New issue   Search
for
  Advanced search   Search tips   Subscriptions
Issue 171: Replace oro.jar regular expressions with java.regex
3 people starred this issue and may be notified of changes. Back to list
Status:  Fixed
Owner:  dmel...@gmail.com
Closed:  Oct 2010


Sign in to add a comment
 
Project Member Reported by dmel...@gmail.com, Jan 6, 2010
What change would like to see?

Migrating over to Java's built-in regular expressions instead of using oro.jar.  Ask Eberhard for 
help.

Why?

So we don't need to continue to include an obsolete regular expression engine.

Would this cause any incompatibilities with previous versions?  If so, how
can these be mitigated?

Jan 9, 2010
#1 e.fa...@wayoda.org
Hi, 

I have been playing with the code a bit, trying to make more suitable for creating an
ubuntu package. In order to get rid of the "oro.jar" I had to rewrite
processing.app.preproc.PdePreprocessor.   

The only noticeable change would be that I created a local version of the
scrubComments(String) method. The original code calls Sketch.scrubComments(String).
But nothing else uses Sketch.scrubComments(String) so why should this not be moved to
the preprocessor class?

Eberhard

PdePreprocessor.java
13.2 KB   View   Download
Jan 10, 2010
#2 e.fa...@wayoda.org
The file I posted previously had 2 bugs.
1) It reported an IllegalArgumentException instead of a RuntimeException when the
user entered unbalanced /* ... */ multiline comments.  
2) It had a problem generating the prototypes when there was a comma in a comment
just before the function definition

Both bugs are fixed in the attached file.

Eberhard 
PdePreprocessor.java
13.4 KB   View   Download
Jan 28, 2010
Project Member #3 dmel...@gmail.com
Can we put scrubComments() back in Sketch.java?  This will make it easier for me to merge any corrections that 
Processing might make to the function, and to provide any changes we make back to Processing.  

Also, it would be great to have more information about the tests you've run on this.  Either the ones I was using 
before <https://code.google.com/p/arduino/source/browse/#svn/branches/junit/app/preproc/test/data> or 
others.
Status: Accepted
Labels: Milestone-0019
Feb 10, 2010
#4 e.fa...@wayoda.org
Here is a new version of the preprocessor. It now passes all the (valid) tests from  
<https://code.google.com/p/arduino/source/browse/#svn/branches/junit/app/preproc/test/data>.
It fails on t14.cpp, but so does the current implementation! 

I would leave scrubComments() in PdePreprocessor. We want a working preprocessor for
Arduino Sketches. If procsessing updates the version in Sketch.scrubComments() it
will very likely incompatible with the requirements of processing Arduino Sketches. 
    
Eberhard
PdePreprocessor.java
14.8 KB   View   Download
Feb 13, 2010
#5 e.fa...@wayoda.org
New Version in reaction to  issue 205 
https://code.google.com/p/arduino/issues/detail?id=205


PdePreprocessor.java
14.8 KB   View   Download
Feb 14, 2010
#6 e.fa...@wayoda.org
And another update caused by  issue205   
Functions can return references too!

PdePreprocessor.java
14.6 KB   View   Download
Feb 19, 2010
Project Member #7 dmel...@gmail.com
Any chance you can submit a more streamlined patch (one that makes the switch from the old oro.jar regular 
expressions to java.regex ones, but without other modifications to the code)?  The other changes may be useful, 
but make it more difficult to understand what's going on here.  Maybe we can implement them separately?
Aug 11, 2010
Project Member #8 dmel...@gmail.com
(No comment was entered for this change.)
Labels: -Milestone-0019
Oct 1, 2010
Project Member #9 c.mag...@bug.st
@dmellis, you can get the patch in attachment, some notes:

1) I can confirm that the patch passes all tests in the old repository (except t14.cpp but i dont know if it's valid).

2) Eberhard make some changes in regexes related to comments:

    // single and multi-line comment
    // p += "|" + "(//\\s*?$)|(/\\*\\s*?\\*/)";             <- E. version
    p += "|(//.*?$)|(/\\*[^*]*(?:\\*(?!/)[^*]*)*\\*/)";     <- actual

but seems that the actual version is the correct one, since the Eberhard one never matches comments.

3) Eberhard made another change in prototype:

"[\\w\\[\\]\\*]+\\s+[&\\[\\]\\*\\w\\s]+\\([&,\\[\\]\\*\\w\\s]*\\)(?=\\s*\\{)"     <- E. version
"[\\w\\[\\]\\*]+\\s+[\\[\\]\\*\\w\\s]+\\([,\\[\\]\\*\\w\\s]*\\)(?=\\s*\\{)"       <- actual

he added the two amps that matches argument passed by reference like:

void func(MyStruct &var);

this one seems correct to me and I keeped it.

4) I've not tested the method firstStatement(...). 
It seems a recently added method that wasn't ported yet. I tried an implementation but is not tested and can (probably) fail. Can you give me some tests to check that?

regexp.patch
7.8 KB   View   Download
Oct 1, 2010
Project Member #10 c.mag...@bug.st
Ok, use the one attached here, this patch solves also the firstStatement(...) issue.
I checked with some examples (using the old method and the new) and both gives the same results.

regexp.patch
7.7 KB   View   Download
Oct 3, 2010
Project Member #11 dmel...@gmail.com
Christian: looks good, I applied it: http://github.com/arduino/Arduino/commit/fa4d0582970e50c574ecfdcdbcfeae3754ca2095


Status: Fixed
Labels: Milestone-0022
Sign in to add a comment

Powered by Google Project Hosting