Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Need to run ./autogen.sh twice #149

Closed
dionyziz opened this issue Dec 31, 2014 · 11 comments
Closed

Need to run ./autogen.sh twice #149

dionyziz opened this issue Dec 31, 2014 · 11 comments

Comments

@dionyziz
Copy link

The first invocation of ./autogen.sh fails, causing ./configure and make to fail. I have to run ./autogen.sh twice. In that case, it works:

root@484cf9b0d581:/home/grr/grr/protobuf# ./autogen.sh
Google Test not present.  Fetching gtest-1.7.0 from the web...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 1136k  100 1136k    0     0  10.0M      0 --:--:-- --:--:-- --:--:-- 11.8M
+ sed -i -e s/RuntimeLibrary="5"/RuntimeLibrary="3"/g;
           s/RuntimeLibrary="4"/RuntimeLibrary="2"/g; gtest/msvc/gtest-md.vcproj gtest/msvc/gtest.vcproj gtest/msvc/gtest_main-md.vcproj gtest/msvc/gtest_main.vcproj gtest/msvc/gtest_prod_test-md.vcproj gtest/msvc/gtest_prod_test.vcproj gtest/msvc/gtest_unittest-md.vcproj gtest/msvc/gtest_unittest.vcproj
+ autoreconf -f -i -Wall,no-obsolete
libtoolize: putting auxiliary files in AC_CONFIG_AUX_DIR, `build-aux'.
libtoolize: copying file `build-aux/config.guess'
libtoolize: copying file `build-aux/config.sub'
libtoolize: copying file `build-aux/install-sh'
libtoolize: copying file `build-aux/ltmain.sh'
libtoolize: putting macros in AC_CONFIG_MACRO_DIR, `m4'.
libtoolize: copying file `m4/libtool.m4'
libtoolize: copying file `m4/ltoptions.m4'
libtoolize: copying file `m4/ltsugar.m4'
libtoolize: copying file `m4/ltversion.m4'
libtoolize: copying file `m4/lt~obsolete.m4'
libtoolize: putting macros in AC_CONFIG_MACRO_DIR, `m4'.
libtoolize: copying file `m4/libtool.m4'
libtoolize: copying file `m4/ltoptions.m4'
libtoolize: copying file `m4/ltsugar.m4'
libtoolize: copying file `m4/ltversion.m4'
libtoolize: copying file `m4/lt~obsolete.m4'
configure.ac:57: installing `./ar-lib'
configure.ac:38: installing `./config.guess'
configure.ac:38: installing `./config.sub'
configure.ac:40: installing `./install-sh'
configure.ac:84: required file `./ltmain.sh' not found
configure.ac:40: installing `./missing'
src/Makefile.am: installing `./depcomp'
autoreconf: automake failed with exit status: 1
root@484cf9b0d581:/home/grr/grr/protobuf# ./autogen.sh
+ sed -i -e s/RuntimeLibrary="5"/RuntimeLibrary="3"/g;
           s/RuntimeLibrary="4"/RuntimeLibrary="2"/g; gtest/msvc/gtest-md.vcproj gtest/msvc/gtest.vcproj gtest/msvc/gtest_main-md.vcproj gtest/msvc/gtest_main.vcproj gtest/msvc/gtest_prod_test-md.vcproj gtest/msvc/gtest_prod_test.vcproj gtest/msvc/gtest_unittest-md.vcproj gtest/msvc/gtest_unittest.vcproj
+ autoreconf -f -i -Wall,no-obsolete
libtoolize: putting auxiliary files in AC_CONFIG_AUX_DIR, `build-aux'.
libtoolize: copying file `build-aux/config.guess'
libtoolize: copying file `build-aux/config.sub'
libtoolize: copying file `build-aux/install-sh'
libtoolize: copying file `build-aux/ltmain.sh'
libtoolize: putting macros in AC_CONFIG_MACRO_DIR, `m4'.
libtoolize: copying file `m4/libtool.m4'
libtoolize: copying file `m4/ltoptions.m4'
libtoolize: copying file `m4/ltsugar.m4'
libtoolize: copying file `m4/ltversion.m4'
libtoolize: copying file `m4/lt~obsolete.m4'
libtoolize: putting auxiliary files in `.'.
libtoolize: copying file `./config.guess'
libtoolize: copying file `./config.sub'
libtoolize: copying file `./install-sh'
libtoolize: copying file `./ltmain.sh'
libtoolize: putting macros in AC_CONFIG_MACRO_DIR, `m4'.
libtoolize: copying file `m4/libtool.m4'
libtoolize: copying file `m4/ltoptions.m4'
libtoolize: copying file `m4/ltsugar.m4'
libtoolize: copying file `m4/ltversion.m4'
libtoolize: copying file `m4/lt~obsolete.m4'
+ rm -rf autom4te.cache config.h.in~
+ exit 0

This occurs in Ubuntu 12.04.

@dionyziz
Copy link
Author

Note that this happens on a clean system only, and can be reproduced using Docker or Travis. If ./autogen.sh is ran after a make clean, it succeeds the first time.

@xfxyjwf
Copy link
Contributor

xfxyjwf commented Jan 7, 2015

I can not reproduce this problem on my machine and our Travis project is also passing: https://travis-ci.org/google/protobuf

Are you using the github master branch or some other version?

@dionyziz
Copy link
Author

dionyziz commented Jan 7, 2015

I'll see if I can make a minimal testcase.

@dionyziz
Copy link
Author

I have a minimal testcase in the autogen_twice branch of my fork. I made two annotated tags on that repository:

  • single, which contains a build file that runs ./autogen.sh once
  • double, which contains a build file that runs ./autogen.sh once

The travis results can be seen in the following links.

Running ./autogen.sh once

When running ./autogen.sh once, the build fails as seen here:
https://travis-ci.org/dionyziz/protobuf/builds/46740409#L270

The error is:

make: *** No targets specified and no makefile found. Stop.

Running ./autogen.sh twice

When running ./autogen.sh twice, the build succeeds as seen here:
https://travis-ci.org/dionyziz/protobuf/builds/46740413#L892

Here is my modified install.sh script which also contains some diagnostic messages. It's just 5 lines, so I consider it minimal.

Here is my modified travis.yml file which simply calls "install.sh".

@xfxyjwf
Copy link
Contributor

xfxyjwf commented Jan 12, 2015

I have found a similar issue report here:
akheron/jansson#182

I tried adding "AC_CONFIG_AUX_DIR([.])" and it seems to fix the problem. Could you check whether it works for you as well?

@dionyziz
Copy link
Author

@xfxyjwf Nope, still failing: https://travis-ci.org/dionyziz/protobuf/builds/47279041

Here's my diff: dionyziz@30e7b27

So I think this is a bug. Interestingly, it's hard to reproduce because you need to run ./autogen.sh on the system twice, but if you run make clean, it "remembers" that you ran autogen previously, so it works with only one run. So you need a really clean system to reproduce the problem.

This seems like two separate problems to me:

  1. autogen.sh runs some things out-of-order and so is required to be ran twice under certain circumstances.
  2. make clean doesn't fully clean everything that autogen.sh left behind, as after you've run autogen.sh followed by a make clean, you no longer need to run it twice.

Interestingly, it may help to note that I cannot reproduce the problem if I don't git clone and use the direct tree provided by travis (as you do in your default .travis.yml). This may point to the right direction for debugging; I suspect it may be related to the particular source tree ./autogen.sh is ran on. However, I cannot pinpoint the problem.

This bug is not blocking me, so no rush. I just have two ./autogen.sh in my build script and it works fine. It just seems like a bug and would be nice to see it fixed, especially because it's so esoteric.

@chrislgarry
Copy link

This is still reproducible. Clean install of Ubuntu 16.04 image on VirtualBox. Protobuf is the very first thing I try to install on this VM, aside from its dependencies and git/vim. Same exact error. ./autogen.sh || autogen.sh is the current work around.

@xfxyjwf
Copy link
Contributor

xfxyjwf commented Jul 20, 2018

@chrislgarry We have never been able to reproduce this issue in our testing environment. It would be great if you can provide a docker image that's able to reproduce the issue.

@chrislgarry
Copy link

chrislgarry commented Jul 21, 2018 via email

@ErichRitz
Copy link

I just hit the same issue: building failed, but running ./autogen.sh twice resulted in a successful build. The weird thing is I have build protobuf3 before and it built successfully. I'm using Slackware and build in a "clean" (full Slackware install with all patches installed) chroot environment using the SlackBuild file from https://slackbuilds.org/repository/14.2/misc/protobuf3/ . In both cases I built protobuf3 release version 3.5.2. So the only difference between my original successful build (May 22nd 2018) and my "need to run autogen.sh twice" build (March 6th 2019) is the Slackware patches I have installed.

A portion of the build log from the successful build:

+ autoreconf -f -i -Wall,no-obsolete
libtoolize: putting auxiliary files in AC_CONFIG_AUX_DIR, 'build-aux'.
libtoolize: copying file 'build-aux/ltmain.sh'
libtoolize: putting macros in AC_CONFIG_MACRO_DIRS, 'm4'.
libtoolize: copying file 'm4/libtool.m4'
libtoolize: copying file 'm4/ltoptions.m4'
libtoolize: copying file 'm4/ltsugar.m4'
libtoolize: copying file 'm4/ltversion.m4'
libtoolize: copying file 'm4/lt~obsolete.m4'
configure.ac:27: installing 'build-aux/compile'
configure.ac:30: installing 'build-aux/config.guess'
configure.ac:30: installing 'build-aux/config.sub'
configure.ac:24: installing 'build-aux/install-sh'
configure.ac:24: installing 'build-aux/missing'
Makefile.am: installing 'build-aux/depcomp'
parallel-tests: installing 'build-aux/test-driver'
libtoolize: putting auxiliary files in AC_CONFIG_AUX_DIR, 'build-aux'.
libtoolize: copying file 'build-aux/ltmain.sh'
libtoolize: Consider adding 'AC_CONFIG_MACRO_DIRS([m4])' to configure.ac,
libtoolize: and rerunning libtoolize and aclocal.
libtoolize: Consider adding '-I m4' to ACLOCAL_AMFLAGS in Makefile.am.
configure.ac:22: installing 'build-aux/compile'
configure.ac:25: installing 'build-aux/config.guess'
configure.ac:25: installing 'build-aux/config.sub'
configure.ac:19: installing 'build-aux/install-sh'
configure.ac:19: installing 'build-aux/missing'
Makefile.am: installing 'build-aux/depcomp'
parallel-tests: installing 'build-aux/test-driver'
libtoolize: putting auxiliary files in '.'.
libtoolize: copying file './ltmain.sh'
libtoolize: putting macros in AC_CONFIG_MACRO_DIRS, 'm4'.
libtoolize: copying file 'm4/libtool.m4'
libtoolize: copying file 'm4/ltoptions.m4'
libtoolize: copying file 'm4/ltsugar.m4'
libtoolize: copying file 'm4/ltversion.m4'
libtoolize: copying file 'm4/lt~obsolete.m4'
configure.ac:66: installing './ar-lib'
configure.ac:61: installing './compile'
configure.ac:46: installing './config.guess'
configure.ac:46: installing './config.sub'
configure.ac:48: installing './install-sh'
configure.ac:48: installing './missing'
benchmarks/Makefile.am: installing './depcomp'
parallel-tests: installing './test-driver'
+ rm -rf autom4te.cache config.h.in~
+ exit 0

A portion of the build log where I was forced to replace "./autogen.sh" with "./autogen.sh || ./autogen.sh" to get a successful build:

+ autoreconf -f -i -Wall,no-obsolete
libtoolize: putting auxiliary files in AC_CONFIG_AUX_DIR, 'build-aux'.
libtoolize: copying file 'build-aux/ltmain.sh'
libtoolize: putting macros in AC_CONFIG_MACRO_DIRS, 'm4'.
libtoolize: copying file 'm4/libtool.m4'
libtoolize: copying file 'm4/ltoptions.m4'
libtoolize: copying file 'm4/ltsugar.m4'
libtoolize: copying file 'm4/ltversion.m4'
libtoolize: copying file 'm4/lt~obsolete.m4'
configure.ac:27: installing 'build-aux/compile'
configure.ac:30: installing 'build-aux/config.guess'
configure.ac:30: installing 'build-aux/config.sub'
configure.ac:24: installing 'build-aux/install-sh'
configure.ac:24: installing 'build-aux/missing'
Makefile.am: installing 'build-aux/depcomp'
parallel-tests: installing 'build-aux/test-driver'
libtoolize: putting auxiliary files in AC_CONFIG_AUX_DIR, 'build-aux'.
libtoolize: copying file 'build-aux/ltmain.sh'
libtoolize: Consider adding 'AC_CONFIG_MACRO_DIRS([m4])' to configure.ac,
libtoolize: and rerunning libtoolize and aclocal.
libtoolize: Consider adding '-I m4' to ACLOCAL_AMFLAGS in Makefile.am.
configure.ac:22: installing 'build-aux/compile'
configure.ac:25: installing 'build-aux/config.guess'
configure.ac:25: installing 'build-aux/config.sub'
configure.ac:19: installing 'build-aux/install-sh'
configure.ac:19: installing 'build-aux/missing'
Makefile.am: installing 'build-aux/depcomp'
parallel-tests: installing 'build-aux/test-driver'
libtoolize: putting auxiliary files in '..'.
libtoolize: copying file '../ltmain.sh'
libtoolize: putting macros in AC_CONFIG_MACRO_DIRS, 'm4'.
libtoolize: copying file 'm4/libtool.m4'
libtoolize: copying file 'm4/ltoptions.m4'
libtoolize: copying file 'm4/ltsugar.m4'
libtoolize: copying file 'm4/ltversion.m4'
libtoolize: copying file 'm4/lt~obsolete.m4'
configure.ac:66: installing './ar-lib'
configure.ac:61: installing './compile'
configure.ac:46: installing './config.guess'
configure.ac:46: installing './config.sub'
configure.ac:48: installing './install-sh'
configure.ac:94: error: required file './ltmain.sh' not found
configure.ac:48: installing './missing'
benchmarks/Makefile.am: installing './depcomp'
parallel-tests: installing './test-driver'
autoreconf: automake failed with exit status: 1
+ autoreconf -f -i -Wall,no-obsolete
libtoolize: putting auxiliary files in AC_CONFIG_AUX_DIR, 'build-aux'.
libtoolize: copying file 'build-aux/ltmain.sh'
libtoolize: putting macros in AC_CONFIG_MACRO_DIRS, 'm4'.
libtoolize: copying file 'm4/libtool.m4'
libtoolize: copying file 'm4/ltoptions.m4'
libtoolize: copying file 'm4/ltsugar.m4'
libtoolize: copying file 'm4/ltversion.m4'
libtoolize: copying file 'm4/lt~obsolete.m4'
configure.ac:27: installing 'build-aux/compile'
configure.ac:24: installing 'build-aux/missing'
Makefile.am: installing 'build-aux/depcomp'
libtoolize: putting auxiliary files in AC_CONFIG_AUX_DIR, 'build-aux'.
libtoolize: copying file 'build-aux/ltmain.sh'
libtoolize: Consider adding 'AC_CONFIG_MACRO_DIRS([m4])' to configure.ac,
libtoolize: and rerunning libtoolize and aclocal.
libtoolize: Consider adding '-I m4' to ACLOCAL_AMFLAGS in Makefile.am.
configure.ac:22: installing 'build-aux/compile'
configure.ac:19: installing 'build-aux/missing'
Makefile.am: installing 'build-aux/depcomp'
libtoolize: putting auxiliary files in '.'.
libtoolize: copying file './ltmain.sh'
libtoolize: putting macros in AC_CONFIG_MACRO_DIRS, 'm4'.
libtoolize: copying file 'm4/libtool.m4'
libtoolize: copying file 'm4/ltoptions.m4'
libtoolize: copying file 'm4/ltsugar.m4'
libtoolize: copying file 'm4/ltversion.m4'
libtoolize: copying file 'm4/lt~obsolete.m4'
configure.ac:61: installing './compile'
configure.ac:48: installing './missing'
benchmarks/Makefile.am: installing './depcomp'
+ rm -rf autom4te.cache config.h.in~
+ exit 0

Note the difference between the two trying to copy ltmain.sh:
Successful build:

libtoolize: putting auxiliary files in '.'.
libtoolize: copying file './ltmain.sh'

autogen.sh twice build:

libtoolize: putting auxiliary files in '..'.
libtoolize: copying file '../ltmain.sh'
...
configure.ac:94: error: required file './ltmain.sh' not found
...
autoreconf: automake failed with exit status: 1
# run autogen.sh a second time
...
libtoolize: putting auxiliary files in '.'.
libtoolize: copying file './ltmain.sh'

So in a failing build it's looking for ltmain.sh in ".." for some reason but in a successful build it looks for ltmain.sh in "."

I have NOT tried reproducing on the latest stable version of protobuf3.

@ErichRitz
Copy link

Ok, I figured out why this is happening. This is what set me on the right track:
https://www.linuxquestions.org/questions/slackware-14/libtoolize-behavior-under-tmp-subdirectories-4175632753/

I (or rather, the build script) was extracting protobuf to /tmp/SBo/protobuf-3.5.2/ and building it there. A different program compilation had (incorrectly) left files directly in /tmp/SBo instead of a subdirectory, namely, /tmp/SBo/install.sh. libtoolize, which is called by autoreconf, which is called by ./autogen.sh during the build, was finding /tmp/SBo/install.sh and assuming /tmp/SBo was the proper build directory instead of /tmp/SBo/protobuf-3.5.2/ (or to be more specific, in relative path terms it was using ".." instead of ".".). Thus ltmain.sh was copied to the "wrong" location and the build failed. Removing /tmp/SBo/install.sh caused the build to be successful.

What I learned: The parent directory of your build directory should be clean of files, especially ones named install.sh. This is not a problem of protobuf, it is a problem of an unclean build environment. Hopefully this helps someone else!

fruffy added a commit to p4gauntlet/gauntlet that referenced this issue Aug 25, 2020
Very esoteric issue: protocolbuffers/protobuf#149
Maybe this fixes the problem. Otherwise we will have to run autogen twice.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants