Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

crash when protocol buffers are used in .so #206

Closed
fkusche opened this issue Feb 11, 2015 · 11 comments
Closed

crash when protocol buffers are used in .so #206

fkusche opened this issue Feb 11, 2015 · 11 comments
Labels
c++ wontfix

Comments

@fkusche
Copy link

fkusche commented Feb 11, 2015

Consider the following foo.cpp:

#include <dlfcn.h>
#include <stdio.h>
#include "p.pb.h"

int main()
{
    void* soHandle = dlopen( "./bar.so", RTLD_NOW );
    if( !soHandle ) {
        fprintf( stderr, "couldn't load bar.so\n" );
        return 1;
    }
    dlclose( soHandle );
    printf( "shutting down protobuf\n" );
    google::protobuf::ShutdownProtobufLibrary();

    return 0;
}

and bar.cpp:

int bar()
{
    return 0;
}

and p.proto:

package FooCfg;
message Msg {
     required int32 dummy = 2;
}

Compile the proto file: protoc --cpp_out=. p.proto
Compile foo to an executable: g++ -fPIC foo.cpp -ofoo -lprotobuf -ldl
Compile bar to a dll: g++ -fPIC -Wl,--unresolved-symbols=ignore-in-shared-libs -shared -o bar.so bar.cpp p.pb.cc -lprotobuf

running foo will produce a segmentation fault in ShutdownProtobufLibrary():

> gdb ./foo
GNU gdb (Debian 7.7.1+dfsg-5) 7.7.1
Copyright (C) 2014 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from ./foo...(no debugging symbols found)...done.
(gdb) r
Starting program: /home/pasu/devel/test/foo 
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
shutting down protobuf

Program received signal SIGSEGV, Segmentation fault.
0x00007ffff6698441 in ?? ()
(gdb) bt
#0  0x00007ffff6698441 in ?? ()
#1  0x00007ffff7af868b in google::protobuf::ShutdownProtobufLibrary() () from /usr/lib/x86_64-linux-gnu/libprotobuf.so.9
#2  0x00000000004008b1 in main ()
(gdb)

Without dlopening bar.so, there will be no segfault.
Also, when p.pb.cc is not compiled into bar.so.

Is this because of global constructors/destructors being executed?
How can this be solved?
@KjellKod
Copy link

@fkusche did you find out what it was? Any work around? I believe a project that I am working on is experiencing a very similar issue.

If we use .so then we see this crash. If we create it as a .a (static) then it doesn't happen.

@fkusche
Copy link
Author

fkusche commented Aug 28, 2015

@KjellKod sorry for the late answer. No I did not find out anything more, unfortunately.

@xfxyjwf
Copy link
Contributor

xfxyjwf commented Jan 20, 2016

As protobuf has used some global variables, linking protobuf dynamically with dlopen across several modules are very problematic. I'm not aware of any good solution here.

@xfxyjwf xfxyjwf closed this as completed Jan 20, 2016
@LocutusOfBorg
Copy link

Hello, if I'm correct this is making the vlc build segfault now
https://trac.videolan.org/vlc/ticket/18329

@DigitAlchemies
Copy link

I encountered this issue when building VLC v3.0.1 using clang/LLVM on Ubuntu 17.10 x86_64 when using protobuf 3.0.0 and found a resolution for the same by taking latest protobuf v3.5.1 source and building it using clang/LLVM and installing, using it instead of protobuf 3.0.0(default one from Ubuntu repo). Here's my complete report for the interested ones : https://trac.videolan.org/vlc/ticket/18329#comment:12

Cheers,
Raghavan Santhanam

kenkuang referenced this issue May 29, 2019
@haberman
Down-integrate from google3.
d61aede
@catskul
Copy link

catskul commented Feb 12, 2020 •
edited

@xfxyjwf IMO this should be reopened. There are plenty of projects that are still having the problem even in 2020. VLC and Gazebo both still have this problem.

Shouldn't this be resovable by adding another global that effectively does a reference count of the loads across modules and refuses to double initialize or double de-initialize the global state?

Probably related: #6959

@catskul catskul mentioned this issue Feb 12, 2020
Closed
@ChristopherHogan ChristopherHogan mentioned this issue Jan 26, 2022
Closed
@csegarragonz csegarragonz mentioned this issue Mar 24, 2022
Merged
@snnn
Copy link
Contributor

snnn commented May 3, 2022

I'm thinking, what if you don't call the "ShutdownProtobufLibrary" function. Libprotobuf does not have any static destructor, so it won't cause problems for you. But if the generated code had any, you need to be sure they are destructed before libprotobuf. In your case, on Linux it is always the case, But on Windows you will need to manually unload the library.

@Rasie1
Copy link

Rasie1 commented Jun 15, 2022

A widely-used library in 2022 still has this gaping issue that stops it from being used in too much situations (e.g. hot reloading). Why is this marked as wontfix?

@fowles
Copy link
Member

fowles commented Jun 19, 2022

.so reloading is incredibly difficult to back into existing libraries that use globals, so the cost of this is quite high. Additionally, we will soon be taking a dependency on abseil in our C++, which also doesn't support dynamic unloading.

@snnn
Copy link
Contributor

snnn commented Jun 20, 2022

which also doesn't support dynamic unloading.

Thanks for giving the hands up. Would you mind to tell us more about this? Absl is a set of static libraries, what blocks this?

@fowles
Copy link
Member

fowles commented Jun 20, 2022

https://abseil.io/about/compatibility is the best summary there (search for "dynamic unloading").

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
c++ wontfix
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
9 participants
@fowles @KjellKod @catskul @snnn @LocutusOfBorg @DigitAlchemies @Rasie1 @xfxyjwf @fkusche