Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

static initialization order - SetupAggressiveDecommit should run after pageheap_ initialization #629

Closed
alk opened this issue Aug 23, 2015 · 2 comments

Comments

@alk
Copy link
Contributor

alk commented Aug 23, 2015

Originally reported on Google Code with ID 626

This issue was first reported in a google groups post:

https://groups.google.com/d/msg/google-perftools/hB-EGfps75k/9rk9xeqR17sJ

What steps will reproduce the problem?

I was building the MongoDB (commit da1f6615489142849764085f5fabfc41f69adbe7) server
against gperftools-2.2 under OS X 10.9.2 and encountered the following crash in the
aggressive decommit static initializer on starting up the server. I was wondering if
you might be able to recommendation a workaround or fix.

Current executable set to './mongod' (x86_64).
(lldb) run
Process 21476 launched: './mongod' (x86_64)
Process 21476 stopped
* thread #1: tid = 0x16f124, 0x00000001007e787b mongod`_GLOBAL__I_a [inlined] tcmalloc::Static::pageheap(this=0x0000000000000000)
at page_heap.h:197, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1,
address=0x8309c)
   frame #0: 0x00000001007e787b mongod`_GLOBAL__I_a [inlined] tcmalloc::Static::pageheap(this=0x0000000000000000)
at page_heap.h:197
  194 
  195   bool GetAggressiveDecommit(void) {return aggressive_decommit_;}
  196   void SetAggressiveDecommit(bool aggressive_decommit) {
-> 197     aggressive_decommit_ = aggressive_decommit;
  198   }
  199 
  200  private:
(lldb) thread backtrace
* thread #1: tid = 0x16f124, 0x00000001007e787b mongod`_GLOBAL__I_a [inlined] tcmalloc::Static::pageheap(this=0x0000000000000000)
at page_heap.h:197, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1,
address=0x8309c)
 * frame #0: 0x00000001007e787b mongod`_GLOBAL__I_a [inlined] tcmalloc::Static::pageheap(this=0x0000000000000000)
at page_heap.h:197
   frame #1: 0x00000001007e787b mongod`_GLOBAL__I_a [inlined] tcmalloc::SetupAggressiveDecommit()
+ 51 at static_vars.cc:120
   frame #2: 0x00000001007e7848 mongod`_GLOBAL__I_a [inlined] tcmalloc::(anonymous
namespace)::google_init_module_tcmalloc_setup_aggressive_decommit() at static_vars.cc:122
   frame #3: 0x00000001007e7848 mongod`_GLOBAL__I_a [inlined] GoogleInitializer(name=0x00007fff5fbffbf0)
+ 40 at googleinit.h:46
   frame #4: 0x00000001007e7820 mongod`_GLOBAL__I_a [inlined] GoogleInitializer at
googleinit.h:47
   frame #5: 0x00000001007e7820 mongod`_GLOBAL__I_a [inlined] __cxx_global_var_init3
+ 108 at static_vars.cc:122
   frame #6: 0x00000001007e77b4 mongod`_GLOBAL__I_a + 4 at static_vars.cc:63
   frame #7: 0x00007fff5fc11c2e

On further investigation, it appears in our build that the static initializer tcmalloc_setup_aggressive_decommit
in static_vars.cc is being invoked before Static::InitStaticVars() is able to initialize
pageheap_.

FYI, the MongoDB builds and links the tcmalloc_minimal object files directly into the
application instead of linking against libtcmalloc_minimal.a or libtcmalloc_minimal.dylib.

git hub url: https://github.com/mongodb/mongo/commit/da1f6615489142849764085f5fabfc41f69adbe7
mongo JIRA: https://jira.mongodb.org/browse/SERVER-8995


What is the expected output? What do you see instead?

In the debugger, I expect to see pageheap_ initialization occur before SetAggressiveDecommit.
Instead, I see SetAggressiveDecommit invoked first and it tries to access a null pageheap_
pointer.

What version of the product are you using? On what operating system?

2.2.
Running this on OS X 2.2 as part of a custom build where I'm linking the object files
directly into the application.

Please provide any additional information below.

Please see response to the issue in the google groups discussion: 

Indeed the SetupAggressiveDecommit should ran after pageheap_ initialization and current
code it not enforcing it, since
InitStaticVars is being called outside the static_vars module.  I think we got lucky
in linker order so tcmalloc_setup_aggressive_decommit
is being called after Static::InitStaticVars.  Check if this quick fix helps your problem:

diff --git a/src/static_vars.cc b/src/static_vars.cc
index 1fa9b40..6d08ca1 100644
--- a/src/static_vars.cc
+++ b/src/static_vars.cc
@@ -96,6 +96,7 @@ void Static::InitStaticVars() {
   // in is caches as pointers that are sources of heap object liveness,
   // which leads to it missing some memory leaks.
   pageheap_ = new (MetaDataAlloc(sizeof(PageHeap))) PageHeap;
+  pageheap_->SetAggressiveDecommit(EnvToBool("TCMALLOC_AGGRESSIVE_DECOMMIT", false));
   DLL_Init(&sampled_objects_);
   Sampler::InitStatics();
 }
@@ -114,11 +115,4 @@ REGISTER_MODULE_INITIALIZER(tcmalloc_fork_handler, SetupAtForkLocksHandler());

 #endif

-static
-void SetupAggressiveDecommit()
-{
-  Static::pageheap()->SetAggressiveDecommit(EnvToBool("TCMALLOC_AGGRESSIVE_DECOMMIT",
false));
-}
-REGISTER_MODULE_INITIALIZER(tcmalloc_setup_aggressive_decommit, SetupAggressiveDecommit());
-
 }  // namespace tcmalloc

Reported by benety.goh@10gen.com on 2014-06-02 13:32:26

@alk
Copy link
Contributor Author

alk commented Aug 23, 2015

merged a fix. Thanks.

Reported by alkondratenko on 2014-06-03 18:15:26

  • Status changed: Fixed

@alk
Copy link
Contributor Author

alk commented Aug 23, 2015

Thank you! We have cherry picked this fix into mongodb/mongo:

https://github.com/mongodb/mongo/commit/8da74f764b055cd610fa41d54c5283faf0986c88

Reported by benety.goh@10gen.com on 2014-06-10 12:21:56

@alk alk closed this as completed Aug 23, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant