Issue 1923: All builds fail
Status:  Invalid
Owner: ----
Closed:  May 2013
Reported by peterbru...@gmail.com, May 21, 2013
************************************************************
***** NOTE: THIS BUG TRACKER IS FOR GERRIT CODE REVIEW *****
***** DO NOT SUBMIT BUGS FOR CHROME, ANDROID, INTERNAL *****
***** ISSUES WITH YOUR COMPANY'S GERRIT SETUP, ETC.    *****
***** THOSE ISSUE BELONG IN DIFFERENT ISSUE TRACKERS!  *****
************************************************************

Affected Version:

What steps will reproduce the problem?
1. git clone https://gerrit.googlesource.com/gerrit
2. git checkout stable-2.6
3. mvn package

What is the expected output? What do you see instead?
1. Expected was a .war file
2. Errors while building the plugins.

Please provide any additional information below.
I would expect that I would be able to build the stable-2.6, 2.7 or master. I tried them all and was unable to. This send me on a few ours long goose chase trying to figure out how it all works.

Now I know that the project.version info in the plugin pom.xml is wrong. This makes the gerrit-war project want to download the plugin jar files that are not present. 

I was able to compile using -Dgerrit.plugins.skip but was then confronted with the new acceptance test that kept on running... 
I have to admit that I am a maven newby but this experience compared to the 2.6rc1 that I compiled a while back is a surprise to me.

I also tried checking out the tags, they also didn't build according to https://gerrit-review.googlesource.com/Documentation/dev-maven.html





May 21, 2013
#3 peterbru...@gmail.com
(No comment was entered for this change.)
build.log
1.7 MB   View   Download
May 21, 2013
Project Member #4 david.pu...@sonymobile.com
As documented in the readme [1], the project should be cloned with --recursive to make sure the git submodules (plugins) are also cloned:

 git clone --recursive https://gerrit.googlesource.com/gerrit

If you've already cloned without --recursive, you can do:

 $ git submodule init
 $ git submodule update

[1] https://gerrit-review.googlesource.com/Documentation/dev-readme.html

May 21, 2013
Project Member #5 david.pu...@sonymobile.com
And building without running the tests is also mentioned in the docs [2]

[2] https://gerrit-review.googlesource.com/Documentation/dev-maven.html#tests
Status: Invalid
May 21, 2013
#6 peterbru...@gmail.com
I think you didn't understand the problem. The plugins were cloned. However, the project.version of these plugins is not the same as the version in the gerrit-war or any of the other core packages. Therefor building any version becomes impossible without excluding plugins.

If the aim is to make gerrit more accessible then it should be possible to just do a simple mvn package without having to have to add -Dthis.doesnt.work=true -Dthat.is.a.problem=true
 
May 21, 2013
#7 sop@google.com
Stop using Maven. :-)
May 21, 2013
#8 peterbru...@gmail.com
What I did in the end to get it to work is replace all <version>2.6-SNAPSHOT</version> with <version>${gerrit-version}</version> and build with mvn package -Dgerrit-version=2.6-test
This way I managed to get the versions consistent in all packages.

Maybe this can be done in general, that way you do not need to update all those pom's just for the version number... 

Or should I start using BUCK? Is that ready for use?

By the way, I still can not build the master, due to codemirror errors. See attached log file.

So current status: stable-2.6 fails due to plugins, stable-2.7 is ok, master fails on codemirror.css :'(

build.log
90.7 KB   View   Download
May 21, 2013
#9 peterbru...@gmail.com
Okay, I guess I will not try buck!

Buck is at 4c5ff21ab6bb1dad6c336b9c8d833e90cfef0868,
but should be c4df74bef4e101a7e5d0176831825b7946ea64a3.
Buck is updating itself.
To disable this, add a '.nobuckcheck' file to your project root.
In general, you should only disable this if you are developing Buck.
fatal: reference is not a tree: c4df74bef4e101a7e5d0176831825b7946ea64a3

May 21, 2013
#10 sop@google.com
You cloned buck from the Facebook tree.
Gerrit has a few patches that are needed to work around open issues in buck:

  git clone https://gerrit.googlesource.com/buck
May 21, 2013
#11 peterbru...@gmail.com
Indeed, I did..

Seems like this issue did result in rapid change from maven to buck in the master. Not that invalid after all ;)

Building with buck now, see how it goes.
May 21, 2013
#12 peterbru...@gmail.com
Build still fails with buck.
build.log
12.8 KB   View   Download
May 21, 2013
#13 sop@google.com
Hmm. Exit status of 247? That's like signal 119... eh? I'm confused.
What is your Hudson environment doing here?
Build works fine with buck, at least outside of Hudson.
May 22, 2013
#14 peterbru...@gmail.com
After the buck updates I get an memory error if I build from the command line:

      [ERROR] Unable to start external process
java.io.IOException: Cannot run program "/usr/lib/jvm/java-6-openjdk-amd64/jre/bin/java": java.io.IOException: error=12, Cannot allocate memory

And the next error when building from Jenkins:
/var/lib/jenkins/buck/bin/buck: line 191: 10621 Killed                  java -XX:MaxPermSize=256m -Xmx1000m $EXTRA_FLAGS -Djava.awt.headless=true -Dbuck.testrunner_classes=${BUCK_DIRECTORY}/build/testrunner/classes -Dbuck.path_to_emma_jar=${BUCK_DIRECTORY}/third-party/java/emma-2.0.5312/out/emma-2.0.5312.jar -

My biggest issue with buck is that it might be faster but when I run it I can't do anything else as my 4 cores are all at 100% for 7-10 minutes. Before with maven I can  run it and check back a while later and still continue to work. This is now not possible anymore.
May 23, 2013
#15 sop@google.com
Maven builds with one thread, Buck by default uses 1.25 * number-of-cores. This usually makes builds faster, but may swamp a machine. You can adjust this setting yourself, e.g.

  buck build --num-threads 1 gerrit

This may also help with the "Cannot allocate memory" error. The GWT compiler needs ~1G of RAM. Buck runs two GWT compilers in parallel by default. Looks like you need to limit to 1 thread.
May 26, 2013
#16 peterbru...@gmail.com
Ok, that worked. I faced one more problem. See the output below:

Traceback (most recent call last):
  File "/var/lib/jenkins/jobs/gerrit-buck/workspace/tools/pack_war.py", line 48, in <module>
    link_jars(args.lib, path.join(war, 'WEB-INF', 'lib'))
  File "/var/lib/jenkins/jobs/gerrit-buck/workspace/tools/pack_war.py", line 38, in link_jars
    cp = check_output(['buck', 'audit', 'classpath'] + libs)
  File "/usr/lib/python2.7/subprocess.py", line 537, in check_output
    process = Popen(stdout=PIPE, *popenargs, **kwargs)
  File "/usr/lib/python2.7/subprocess.py", line 679, in __init__
    errread, errwrite)
  File "/usr/lib/python2.7/subprocess.py", line 1249, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory


This was caused by the fact that I cloned buck and build it on /var/lib/jenkins/buck and not added it to the /usr/bin. At some point the pack_war.py performs a call back to buck and then buck can not be found. 

Anyway, thanks for your help. With the num-threads set to one the memory problem and high cpu usage is gone.

Maybe the num-threads and the relation to running out of memory could be added to the manual.