Comment 1:

http://build.golang.org/log/0b2bfadd4666b7ddd040c4ae3fd1c35a4a715fbd
http://build.golang.org/log/265eb38fb47999a2a7ee16cf340107a94cc79690
http://build.golang.org/log/00accd8695013e0ce9325516a51cecc391ddd0df
http://build.golang.org/log/3d01c97ce75337ec524d90e3ea43469cdb24e78f

Labels changed: added arch-arm, os-linux, cgo.

Comment 2:

i think we can modify doc/progs/run to output some progress indication,
and then try to reproduce the problem. I don't quite understand what is
triggering SEGSEGV here.

Comment 3:

SGTM. If you prepare the CL I can run doc/progs in a loop til it breaks.

Comment 4:

just add a line "set -x" to doc/prog/run.

Comment 5:

I believe this is not a problem with go code, but a segfault in 5g or
5l, as doc/progs never runs the code, it only compiles.
On this machine CC is set to /usr/bin/clang-3.0, so it could be a clang bug.
I have been running doc/progs/run in a loop for two hours now without
a crash, this might be quiet subtle.

Comment 6:

maybe we can modify cmd/go to display the exact command that is failing?
btw, doc/progs/run does run some of the compiled binaries and test
their outputs.

Comment 7:

Since replacing the linux/arm builder in question, there have been no incidents. I will
keep this issue open for another few weeks, then close if nothing shows up.

Comment 8:

I'm putting this one down to the Pandaboard not being awesome.

Status changed to Retracted.

Comment 9:

http://build.golang.org/log/00600fa74b379cb566c2b63ae5736e6f06aa9a98

Status changed to Accepted.

Comment 10:

Removing myself as a owner.
As a datapoint the current linux-arm-cheney builder, an omap3 freescale iMX53 board has
never failed a build in this manner -- however this is a single core machine. 
I am considering running ./run.bash in a loop on my nexus7 (if I can figure out a way to
minimise the flash writes) to gather another data point on if this is caused by a data
race, or if it is just pandaboards being flaky under load.

Owner changed to ---.

Comment 11:

I have been running variations of ./run.bash --no-rebuild, ./run.bash, and ./all.bash in
a loop on my pandaboard and nexus 7 for the last 24 hours at revision
% hg id /tmp/go
019884311591+ tip
without incident.
@minux, are you able to try again with your pandaboard ?

Comment 12:

Labels changed: added priority-later, removed priority-triage.

Comment 13:

Lowering the priority to go1.1maybe, this issue has not reoccured since pandaboards were
removed from the equation.

Labels changed: added go1.1maybe, removed go1.1.

Comment 14:

After some more debugging, it appears the common factor is at a minimum, Ubuntu Linux
11.10 - 12.04 and a pandaboard. Replacing the operating system with arch linux has
produced a builder which has performed flawlessly in soak tests and proven to be a very
stable builder. 
The remaining question is what in the Ubunut linux + pandaboard combination causes the
strange segfaults. For the moment I will add a note to the builder page that ubuntu
11.10/12.04 OMAP4 kernels are not recommended as builders.

Comment 15:

i want to do the following experiment:
1. patch cmd/dist to build static binary
2. compile on affected ubuntu
3. copy whole $GOROOT to arch
4. do extensive test on arch with the $GOROOT.
I hope this could isolate the problem to either
libc/toolchain, or the kernel.
In the past, I do see broken gcc that miscompile
our compiler.

Comment 16:

I am very confident that we have isolated the fix -- older versions of the ubuntu OMAP4
kernel, and demonstrated that the fix is to apt-get upgrade to the latest available
kernel (12.04.2). 
I am marking this as fixed, and sincerely hope it will not be reopened again.

Status changed to Fixed.