Skip to content
This repository has been archived by the owner on Apr 21, 2023. It is now read-only.

setting malloc/free/realloc #483

Open
GoogleCodeExporter opened this issue Apr 6, 2015 · 5 comments
Open

setting malloc/free/realloc #483

GoogleCodeExporter opened this issue Apr 6, 2015 · 5 comments

Comments

@GoogleCodeExporter
Copy link

Hi,

Is it currently possible to specify memory allocation functions through code 
for pagespeed automatic? It would be very nice if I can let the traffic server 
port use the allocation routines that trafficserver provides. 

ps. we are planning to open source the port very soon, we will donate the code 
to apache. So you can expect it to turn up soon around here:  
https://github.com/apache/trafficserver/tree/master/plugins/experimental
:) 

Cheers,

Otto van der Schaaf

Original issue reported on code.google.com by osch...@gmail.com on 30 Jul 2012 at 9:05

@GoogleCodeExporter
Copy link
Author

For the most part, we use operator new/delete/[]new/[]delete to allocate 
memory.  Overrides for those operators can be linked in as desired, though be 
aware that src/net/instaweb/util/mem_debug.cc already does that in debug-mode 
only.

There may be a few library routines that we call that use malloc/free/delete 
directly.  Is it essential that 100% of the allocations go through the 
trafficserver allocator?   Or would it still be beneficial if only 99% went 
through the standard x-ray system, with the rest leaking into malloc/free?

Original comment by jmara...@google.com on 7 Aug 2012 at 5:05

  • Changed state: RequestClarification

@GoogleCodeExporter
Copy link
Author

Hi.  I just checked 
https://github.com/apache/trafficserver/tree/master/plugins/experimental and 
still did not see your ATS plugin.  How is that going?

Do you have any clarification for my questions about your memory allocator 
requirements?

Original comment by jmara...@google.com on 5 Sep 2012 at 2:36

@GoogleCodeExporter
Copy link
Author

- I just replied to your email regarding the plugin.  

I couldn't get your suggestion to work at the time, and haven't looked into it 
since; i have been distracted.
I guess 99% would be okay, but to i'd have to inquire some people better suited 
then me to judge which allocations would be most important to catch here. the 
nice thing about it, is that in turn traffic server allows choosing an 
allocator at configure time. For example jemalloc 
http://www.facebook.com/notes/facebook-engineering/scalable-memory-allocation-us
ing-jemalloc/480222803919



Original comment by osch...@gmail.com on 5 Sep 2012 at 9:52

@GoogleCodeExporter
Copy link
Author

I think we should consider:

a) having a reasonable way for PSOL API users to set the allocator
b) overriding the allocator in the mod_pagespeed binary
c) overriding the allocator in the ngx_pagespeed build instructions

jemalloc is one possibility
tcmalloc is another.  I think we need to benchmark.

This may be associated with an apparent leak as discussed here: 
https://groups.google.com/forum/?utm_medium=email&utm_source=footer#!msg/mod-pag
espeed-discuss/NjGRuTRGPyo/TvcudyMdcjUJ
It's possible that this leak is due to fragmentation, and that switching to a 
better memory allocator will solve it.

It's also possible that we could use pooling more/better in the parser and that 
would also solve it.  Conversely we may be *causing* fragmentation issues with 
the pooling that we *are* doing.

It also may be associated with load-test performance on a pure HTML load as 
discussed here:
https://groups.google.com/forum/#!topic/mod-pagespeed-discuss/HyIBCaBC_Es

Original comment by jmara...@google.com on 23 Oct 2014 at 2:55

  • Changed state: New

@GoogleCodeExporter
Copy link
Author

jemalloc discussion on chromium: 
https://groups.google.com/a/chromium.org/forum/#!topic/chromium-dev/EUqoIz2iFU4

Original comment by jmara...@google.com on 10 Dec 2014 at 3:53

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

1 participant