I got similar issue:
My project started from GWT 1.33 until latest 1.53. All works well. 
But 1.6RC can not compile. It looks an infinite looping.  I already 
set xmx to 512M. 
Exception from here, I ignore thousand lines... 
   [java] Compiling module com.edgenius.wiki.gwt.PageApplication 
     [java]    [ERROR] Unexpected internal compiler error 
     [java] java.lang.StackOverflowError 
     [java]     at java.lang.System.arraycopy(Native Method) 
     [java]     at sun.reflect.ByteVectorImpl.resize(ByteVectorImpl.java: 
67) 
     [java]     at sun.reflect.ByteVectorImpl.add(ByteVectorImpl.java:45) 
     [java]     at sun.reflect.ClassFileAssembler.emitByte 
(ClassFileAssembler.java:56) 
     [java]     at sun.reflect.ClassFileAssembler.emitConstantPoolUTF8 
(ClassFileAssembler.java:89) 
     [java]     at sun.reflect.MethodAccessorGenerator.generate 
(MethodAccessorGenerator.java:276) 
     [java]     at 
sun.reflect.MethodAccessorGenerator.generateSerializationConstructor 
(MethodAccessorGenerator.java:95) 
     [java]     at 
sun.reflect.ReflectionFactory.newConstructorForSerialization 
(ReflectionFactory.java:313) 
     [java]     at java.io.ObjectStreamClass.getSerializableConstructor 
(ObjectStreamClass.java:1299) 
     [java]     at java.io.ObjectStreamClass.access$1500 
(ObjectStreamClass.java:52) 
     [java]     at java.io.ObjectStreamClass$2.run(ObjectStreamClass.java: 
420) 
     [java]     at java.security.AccessController.doPrivileged(Native 
Method) 
     [java]     at java.io.ObjectStreamClass.<init> 
(ObjectStreamClass.java:400) 
     [java]     at java.io.ObjectStreamClass.lookup 
(ObjectStreamClass.java:297) 
     [java]     at java.io.ObjectOutputStream.writeObject0 
(ObjectOutputStream.java:1035) 
     [java]     at java.io.ObjectOutputStream.defaultWriteFields 
(ObjectOutputStream.java:1375) 
     [java]     at java.io.ObjectOutputStream.writeSerialData 
(ObjectOutputStream.java:1347) 
     [java]     at java.io.ObjectOutputStream.writeOrdinaryObject 
(ObjectOutputStream.java:1290) 
     [java]     at java.io.ObjectOutputStream.writeObject0 
(ObjectOutputStream.java:1079) 
     [java]     at java.io.ObjectOutputStream.defaultWriteFields 
(ObjectOutputStream.java:1375) 
     [java]     at java.io.ObjectOutputStream.writeSerialData 
(ObjectOutputStream.java:1347) 
     [java]     at java.io.ObjectOutputStream.writeOrdinaryObject 
(ObjectOutputStream.java:1290) 
     [java]     at java.io.ObjectOutputStream.writeObject0 
(ObjectOutputStream.java:1079) 
     [java]     at java.io.ObjectOutputStream.writeObject 
(ObjectOutputStream.java:302) 
     [java]     at java.util.ArrayList.writeObject(ArrayList.java:569) 
     [java]     at sun.reflect.GeneratedMethodAccessor57.invoke(Unknown 
Source) 
     [java]     at sun.reflect.DelegatingMethodAccessorImpl.invoke 
(DelegatingMethodAccessorImpl.java:25) 
     [java]     at java.lang.reflect.Method.invoke(Method.java:585) 
     [java]     at java.io.ObjectStreamClass.invokeWriteObject 
(ObjectStreamClass.java:917) 
     [java]     at java.io.ObjectOutputStream.writeSerialData 
(ObjectOutputStream.java:1339) 
     [java]     at java.io.ObjectOutputStream.writeOrdinaryObject 
(ObjectOutputStream.java:1290) 
     [java]     at java.io.ObjectOutputStream.writeObject0 
(ObjectOutputStream.java:1079) 
     [java]     at java.io.ObjectOutputStream.defaultWriteFields 
(ObjectOutputStream.java:1375) 
     [java]     at java.io.ObjectOutputStream.writeSerialData 
(ObjectOutputStream.java:1347) 
     [java]     at java.io.ObjectOutputStream.writeOrdinaryObject 
(ObjectOutputStream.java:1290) 
     [java]     at java.io.ObjectOutputStream.writeObject0 
(ObjectOutputStream.java:1079) 
     [java]     at java.io.ObjectOutputStream.defaultWriteFields 
(ObjectOutputStream.java:1375) 
     [java]     at java.io.ObjectOutputStream.writeSerialData 
(ObjectOutputStream.java:1347) 
     [java]     at java.io.ObjectOutputStream.writeOrdinaryObject 
(ObjectOutputStream.java:1290) 
     [java]     at java.io.ObjectOutputStream.writeObject0 
(ObjectOutputStream.java:1079) 
     [java]     at java.io.ObjectOutputStream.defaultWriteFields 
(ObjectOutputStream.java:1375) 
     [java]     at java.io.ObjectOutputStream.writeSerialData 
(ObjectOutputStream.java:1347) 
     [java]     at java.io.ObjectOutputStream.writeOrdinaryObject 
(ObjectOutputStream.java:1290) 
     [java]     at java.io.ObjectOutputStream.writeObject0 
(ObjectOutputStream.java:1079) 
     [java]     at java.io.ObjectOutputStream.defaultWriteFields 
(ObjectOutputStream.java:1375) 
     [java]     at java.io.ObjectOutputStream.writeSerialData 
(ObjectOutputStream.java:1347) 
     [java]     at java.io.ObjectOutputStream.writeOrdinaryObject 
(ObjectOutputStream.java:1290) 
     [java]     at java.io.ObjectOutputStream.writeObject0 
(ObjectOutputStream.java:1079) 
     [java]     at java.io.ObjectOutputStream.defaultWriteFields 
(ObjectOutputStream.java:1375) 
     [java]     at java.io.ObjectOutputStream.writeSerialData 
(ObjectOutputStream.java:1347) 
     [java]     at java.io.ObjectOutputStream.writeOrdinaryObject 
(ObjectOutputStream.java:1290) 
     [java]     at java.io.ObjectOutputStream.writeObject0 
(ObjectOutputStream.java:1079) 
     [java]     at java.io.ObjectOutputStream.defaultWriteFields 
(ObjectOutputStream.java:1375) 
     [java]     at java.io.ObjectOutputStream.writeSerialData 
(ObjectOutputStream.java:1347) 
     [java]     at java.io.ObjectOutputStream.writeOrdinaryObject 
(ObjectOutputStream.java:1290) 
     [java]     at java.io.ObjectOutputStream.writeObject0 
(ObjectOutputStream.java:1079) 
     [java]     at java.io.ObjectOutputStream.defaultWriteFields 
(ObjectOutputStream.java:1375) 
     [java]     at java.io.ObjectOutputStream.writeSerialData 
(ObjectOutputStream.java:1347) 
     [java]     at java.io.ObjectOutputStream.writeOrdinaryObject 
(ObjectOutputStream.java:1290) 
     [java]     at java.io.ObjectOutputStream.writeObject0 
(ObjectOutputStream.java:1079) 
     [java]     at java.io.ObjectOutputStream.defaultWriteFields 
(ObjectOutputStream.java:1375) 
     [java]     at java.io.ObjectOutputStream.writeSerialData 
(ObjectOutputStream.java:1347) 
     [java]     at java.io.ObjectOutputStream.writeOrdinaryObject 
(ObjectOutputStream.java:1290) 
     [java]     at java.io.ObjectOutputStream.writeObject0 
(ObjectOutputStream.java:1079) 
     [java]     at java.io.ObjectOutputStream.defaultWriteFields 
(ObjectOutputStream.java:1375) 
     [java]     at java.io.ObjectOutputStream.writeSerialData 
(ObjectOutputStream.java:1347) 
     [java]     at java.io.ObjectOutputStream.writeOrdinaryObject 
(ObjectOutputStream.java:1290) 
     [java]     at java.io.ObjectOutputStream.writeObject0 
(ObjectOutputStream.java:1079) 
     [java]     at java.io.ObjectOutputStream.defaultWriteFields 
(ObjectOutputStream.java:1375) 
     [java]     at java.io.ObjectOutputStream.writeSerialData 
(ObjectOutputStream.java:1347) 
     [java]     at java.io.ObjectOutputStream.writeOrdinaryObject 
(ObjectOutputStream.java:1290) 
     [java]     at java.io.ObjectOutputStream.writeObject0 
(ObjectOutputStream.java:1079) 
     [java]     at java.io.ObjectOutputStream.writeObject 
(ObjectOutputStream.java:302) 
     [java]     at java.util.ArrayList.writeObject(ArrayList.java:569) 
     [java]     at sun.reflect.GeneratedMethodAccessor57.invoke(Unknown 
Source) 
..................  ignore similar 
     [java]     at java.io.ObjectStreamClass.invokeWriteObject 
(ObjectStreamClass.java:917) 
     [java]     at java.io.ObjectOutputStream.writeSerialData 
(ObjectOutputStream.java:1339) 
     [java]     at java.io.ObjectOutputStream.writeOrdinaryObject 
(ObjectOutputStream.java:1290) 
     [java]     at java.io.ObjectOutputStream.writeObject0 
(ObjectOutputStream.java:1079) 
     [java]     at java.io.ObjectOutputStream.defaultWriteFields 
(ObjectOutputStream.java:1375) 
     [java]     at java.io.ObjectOutputStream.writeSerialData 
(ObjectOutputStream.java:1347) 
     [java]     at java.io.ObjectOutputStream.writeOrdinaryObject 
(ObjectOutputStream.java:1290) 
     [java]     at java.io.ObjectOutputStream.writeObject0 
(ObjectOutputStream.java:1079) 
     [java]     at java.io.ObjectOutputStream.defaultWriteFields 
(ObjectOutputStream.java:1375) 
     [java]     at java.io.ObjectOutputStream.writeSerialData 
(ObjectOutputStream.java:1347) 
     [java]     at java.io.ObjectOutputStream.writeOrdinaryObject 
(ObjectOutputStream.java:1290) 
     [java]     at java.io.ObjectOutputStream.writeObject0 
(ObjectOutputStream.java:1079) 
     [java]     at java.io.ObjectOutputStream.defaultWriteFields 
(ObjectOutputStream.java:1375) 

@Scott: does this look like something you might have seen before?

@karthikeyan + dapeng: Do you know if the compilation error occurs when restricting 
the user agent to another other singular browser target? Or does it only occur when a 
specific user-agent isn't specified for compilation?

Thanks for the reply Scott. I generally do my compilation without specifying
user-agent, since i got this message I was trying out some options considering it
could be memory issue, one of the thing i tried is setting user.agent to be ie6
alone, and it worked without this compilation error.

In gwt 1.5.3 all works without specifying user.agent property too.

I tried passing firefox target alone it failed. But havent tried with safari or opera
options.

 

This is a plain old Java stack overflow... in 1.6 we serialize/deserialize large 
object graphs which takes more stack than was needed in GWT 1.5.

If your project is failing, try using the "-Xss" JVM option to give the compiler more 
stack size.

Thanks Scott, adding Xss has fixed it.
<jvmarg value="-Xss1024k"/> 

@dapeng: you're likely facing the same problem. Please give Scott's suggestion a try 
to see if it resolves the issue.

Marking issue as Invalid.

Actually, I would like to make AST serialization take less stack, so let's leave this 
open.

GWT version: 1.6.3
Helped only on one computer, with java 1.6 installed(Windows). If I try to compile
with java 1.5.0_09(Windows) on other computer it fails StackOverflowError even if I
set -Xss100M, if I set more than 120Mb it throws OutOfMemoryException. 

Also I've tried to compile with java 1.5.0_16 on other(third) computer(Centos4.4) and
compilation succeded.

Any ideas what can be wrong with java 1.5.0_9+GWT1.6.3+WindowsXP? 

I have the same issue, our project compiles on gwt 1.5.3 but on gwt 1.6.3 throws
unexpected compiler exception, we tried with Xss but nothing.

     [java]    [ERROR] Unexpected internal compiler error
     [java] java.lang.StackOverflowError
     [java] 	at java.io.ObjectStreamClass.processQueue(ObjectStreamClass.java:2206)
     [java] 	at java.io.ObjectStreamClass.lookup(ObjectStreamClass.java:253)
     [java] 	at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1035)
     [java] 	at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:302)
     [java] 	at java.util.ArrayList.writeObject(ArrayList.java:569)
     [java] 	at sun.reflect.GeneratedMethodAccessor57.invoke(Unknown Source)

 

same issue whith gwt 1.6.x.
Problem seems to come from firefox compilation... If i restrict user.agent to
firefox, i get compiler exception; even if i increase Xss.
Restricting user.agent to ie6 works fine.

I am having this problem in 1.6.4, on a project that compiles correctly under 1.5.2.

The stack trace consists mostly of repetitions of:
	at java.io.ObjectOutputStream.defaultWriteFields(Unknown Source)
	at java.io.ObjectOutputStream.writeSerialData(Unknown Source)
	at java.io.ObjectOutputStream.writeOrdinaryObject(Unknown Source)
	at java.io.ObjectOutputStream.writeObject0(Unknown Source)

It also has sections like:

	at java.io.ObjectOutputStream.writeObject(Unknown Source)
	at java.util.ArrayList.writeObject(Unknown Source)
	at sun.reflect.GeneratedMethodAccessor44.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
	at java.lang.reflect.Method.invoke(Unknown Source)
	at java.io.ObjectStreamClass.invokeWriteObject(Unknown Source)
	at java.io.ObjectOutputStream.writeSerialData(Unknown Source)
	at java.io.ObjectOutputStream.writeOrdinaryObject(Unknown Source)
	at java.io.ObjectOutputStream.writeObject0(Unknown Source)

It does not seem to matter if I target a specific browser.

The error message with the complete stack trace is attached.

	FullErrorMessage.txt
	63.8 KB Download

The argument to -Xss is actually a stack size.  You can set it even larger than 1024k.  If -Xss1024k gets stack 
overflows, please try with 2048k or 4000k.

Increasing the stack size fixed the problem for me, though I appreciate ScottB's
desire to make the process use less stack.

Also, I somehow missed ScottB's very relevant comment #4 when reading this issue the
first time. It might be useful to be able to tag comments as "workaround/solution" to
make them stand out.

Maybe changing the subject will help a bit.

No luck.

I've tried up to 50M Xss. Above that, I run into OutOfMemoryError's.  

If I build with 1 permutation (FF only), it completes successfully. If I run against
all 5 permutations, I get a StackOverFlow error always. I have not tried targeting
other permutations specifically. 

My compile command: 
java -Xss50M -Xmx1000M -Xms64M -cp %CLASSPATH%  com.google.gwt.dev.Compiler -war
%OUT_DIR%

	stack.txt
	70.9 KB Download

I got 1.6.4, it still no luck. 
tried Xss to 56M, still get java.lang.StackOverflowError

I have a fix for this.  I'd like to actually roll it into any potential 1.6 updates 
we do, so it's against 1.6.  It would probably require some manual massaging to port 
to trunk from here.

	issue-3510.patch
	8.2 KB Download

Follow-up to my previous comment #15:

I managed to get it to compile. Turns out, my compilation script was defaulting to 
JDK 1.5. By switching my PATH to use JDK 1.6.0_10, I'm able to compile.

What I've noticed:
- On our 64bit Win2k3 system w/ JDK 1.5.x, even -Xss50M is not sufficient
- On our 64bit Win2k3 system w/ JDK 1.6.0_10, the only parameter I need is: -Xmx1200M
- On our 32bit WinXp & Vista systems, we need to specify -Xss32M. Not sure if 32M is
really needed, but it worked. 

Summary: Make sure you're on JDK 1.6

I had this problem and was already using Java 6.0

I followed the suggestion above to add

<jvmarg value="-Xss1024k"/> 

and I can compile successfully with Ant now. But the Eclipse plugin for compiling and 
the compile button the hosted mode server both get stack overflows.

 Issue 2794  has been merged into this issue.

Anyone want to try this patch jar and see if it both fixes your issue and compiles 
successfully?  Ideally, you could remove the -Xss spec and still get a clean compile.  
Just place this jar in your classpath higher than gwt-dev-<platform>.jar.

Note that this will only likely work with GWT 1.6.4.

	gwt-dev-patch.jar
	30.1 KB Download

I'm trying to migrate my project from GWT 1.5.3 to GWT 1.6.4:

OS: Windows XP, 32 bit
GWT: 1.6.4 (svn revision 1.6@5189)
Java: java version "1.6.0_13"
JVM Args: -Xms512M -Xmx1024M

I'm using Maven2 with com.totsp.gwt:maven-googlewebtoolkit2-plugin for building. This
plugin loads GWT compiler directly from GWT home directory, so gwt-dev-patch JAR file
had to be merged into gwt-dev-windows JAR file (instead of simply prepending it to
the dependencies list).

Before gwt-dev-patch: 2 modules out of 3 failed with StackOverflowError, even when
the extra JVM arg -Xss64M was set
After gwt-dev-patch: 3 modules out of 3 succeeded. No need for the extra JVM arg -Xss.

That's terrific news.

It should also compile faster too, no? 

Scottb, I had the same problem and the patch has worked really good for me! Thank you!

i did update our app from 1.3 to 1.5 and 1.6.. on JDK5 and JDK6 got
java.lang.StackOverflowError and also outOfMemory errors...but with patch compiles
OK, even without Xss flag!
my build settings:
<java classpathref="gwt.classpath" classname="com.google.gwt.dev.Compiler" fork="true">
				<arg value="-logLevel" />
				<arg value="INFO" />
				<arg value="-gen" />
				<arg value="tmp" />
				<arg value="-war" />
				<arg value="@{output-dir}" />
				<arg value="@{module}" />
				
				<jvmarg value="-Xmx512M"/>				
				<jvmarg value="-Xms256m"/>				
			</java>

have to admit it was a large app and in 1.3.3 style, so many raw types

     [java] Compiling module xxxxx
     [java]    [ERROR] Unexpected internal compiler error
     [java] java.lang.StackOverflowError
     [java]     at
java.io.ObjectOutputStream$BlockDataOutputStream.write(ObjectOutputStream.java:1802)
     [java]     at
java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1495)
     [java]     at
java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1474)
     [java]     at
java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1392)
     [java]     at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1150)
     [java]     at
java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1509)
     [java]     at
java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1474)
     [java]     at
java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1392)
     [java]     at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1150)
     [java]     at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:326)
     [java]     at java.util.ArrayList.writeObject(ArrayList.java:570)
     [java]     at sun.reflect.GeneratedMethodAccessor55.invoke(Unknown Source)
     [java]     at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)

@Scott, the patch LGTM.

(No comment was entered for this change.)

Thanks, Bob.  Committed to 1.6 branch @r5262.

 Issue 3585  has been merged into this issue.

(No comment was entered for this change.)

Wonderfull

Congratulations from Brazil to Scott! (post 21)

Is there a scheduled release for this?

(No comment was entered for this change.)

Great.

Congratulations from Chile to Scott! (post 21)

just want to add that the patch by scott worked for me too, thx!

@scottb

Hi, Scott,

I read the topic about the stack overflow and I tried your solution, but I had no
success on it. Could you help me?

I downloaded "gwt-dev-patch", and add to <my_project\war\WEB-INF\lib> and then I
opened <my_project> -> properties -> Libraries -> Add JARs and added the jar you made.

My .classpath got this extra line: 	
<classpathentry kind="lib" path="war/WEB-INF/lib/gwt-dev-patch.jar"/>

After that, I ran the project and clicked on Compile/Browser... I got a message: "GWT
module 'index' needs to be (re)compiled, please run a compile or use the
Compile/Browse button in hosted mode"

At Google Hosted Mode, the error:
"
[ERROR] Unexpected internal compiler error
java.lang.StackOverflowError: null
	at java.io.ObjectStreamClass.lookup(ObjectStreamClass.java:244)
	at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1029)
	at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1369)
	at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1341)
[...]
"

It is the same error, you, guys, discussed on this topic. I wasn't able to use -Xss,
I think it is due to my application, it is too large.

If you have any ideas....

Thanks
Eduardo

@scott

Just adding...

I tried to remove gwt-dev-windows for my gwt libs and I couldn't manage the project
'cause of other libs on gwt-dev-windows original. 

Then I opened the original one and add the code which you wrote into the original
one, and I couldn't compile it.

Well, this is it.

Thanks

Edaurdo: don't but the patch jar in WEB-INF/lib, put it somewhere else, like your GWT 
install directory.  Also, you need to make sure on the classpath that the patch jar 
comes JUST BEFORE gwt-dev-windows.  So it should look like this in your .classpath 
file:

<classpathentry kind="lib" path="path/to/gwt/gwt-dev-patch.jar"/>
<classpathentry kind="lib" path="path/to/gwt/gwt-dev-windows.jar"/>

@scott

Sorry for all this posts! I put it to work! 

Thanks

humm

I will try this one. 'Cause I put the gw-dev-patch content into gwt-dev-windows. It
gave me a little work, but I got it working. ^^

Thanks a lot

thanks scottb. it works!

Great! Thanks scott. it works.

Thanks a lot, it works perfectly

Hey its working fine.. I was having the same problem and now it has been solved by
using the patch. I am working on a GWT GXT application using eclipse plug-inns. After
adding the gwt-dev-patch.jar file with my GWT library files everything works fine. In
eclipse at Window -> Preferences -> Java -> Build Path -> User Libraries -> I added
the Patch to my GWT library and also moved the position of gwt-dev-patch.jar just
above gwt-dev-windows.jar.
Now when I clicked the Compile Button I get:
Compiling module XXXXX
   Compiling 5 permutations
      Permutation compile succeeded
   Linking into war
      Link succeeded
   Compilation succeeded -- 43.813s

Cheers..!!

Thanks Scott.

Thanks, works for me too.  

I am using maven plugin com.totsp.gwt.maven-googlewebtoolkit2-plugin and had to merge
the patch jar with my gwt-dev-windows.jar but can now build from command line (mvn
gwt:compile) or using the compile button in hosted mode browser.  Thanks, again.

Patch solved problem here too. Thanks.

Awesome ! We were desesperate by this stack overflow exception and the patch solved
our problem. 
Best regards.

It also works for me, thank you.

works for me, I use maven with com.totsp.gwt.maven-googlewebtoolkit2-plugin, make
sure the patch on top of other classpath.

Hi, I try all you suggestions for risolve the same problem but without results.
I follow Comment 46:
- download patch
- In inside the path gwt-windows-1.6.4 I added the gwt-dev-patch.jar
- In eclipse : Window -> Preferences -> Java -> Build Path -> User Libraries -> 
created
  a new User Libray with inside all library of GWT and the patch.
- Added the user library betwean the library of the project
- Click button GWT Compile Project

The error still to be

Really thanks for this patch. This have solved compiling issues on my very first real
appplication. I'm using GWT 1.6.4 + GXT 2.0 M3 + GWT-XMLRPC + GWT-LOG
Could we have more information about this patch and when it should be integrated in
official release ?

Scott you are the BEST!!!!!!!!!!!!!!!!

Thanks guys, I'm going to lock this issue down now to make it easier for people to find 
what's important.

Options for getting this fix:
1) See comment #21 for a jar you can use with GWT 1.6.4.
2) Build GWT's releases/1.6 branch yourself from source.
3) Wait for GWT 1.6.5 to be released.

This shipped in GWT 1.7.0.