Skip to content
This repository has been archived by the owner on Apr 21, 2023. It is now read-only.

apache2 coredump after enabling mod_pagespeed's extend_cache on centOS. #610

Closed
GoogleCodeExporter opened this issue Apr 6, 2015 · 9 comments
Closed

apache2 coredump after enabling mod_pagespeed's extend_cache on centOS. #610

GoogleCodeExporter opened this issue Apr 6, 2015 · 9 comments
Labels
auto-migrated Milestone-v25 Priority-Medium Type-Defect

Comments

@GoogleCodeExporter
Copy link 

This is a cross post from my question on serverfault: 
http://serverfault.com/questions/461448/apache2-coredump-after-enabling-mod-page
speed-on-centos

I'm trying to get the following apache2.2 modules to work together 
mod_pagespeed, mod_spdy and WebSphere Webserver plugin on CentOS.

Once mod_pagespeed changes the path of a file the browser can't resolve that 
file anymore. Enable or disable mod_spdy doesn't change any behaviour. 
mod_pagespeed with extend_cache works (no coredumps) if I disable WebSphere 
plugin. 


Response from Eric Covener:
it looks like a mod_pagespeed bug (null pointer dereference) hen r->filename is 
not set by the handler. mod_proxy sets it to proxy:$originaluri, the WAS Plugin 
doesn't set it it at all.

I have a coredump but I don't really know how to read this:
gdb /usr/sbin/httpd core.881
(gdb) bt full
#0 __strrchr_sse42 () at ../sysdeps/x86_64/multiarch/strrchr.S:134
No locals.
#1 0x00007ff129b02404 in ?? () from /usr/lib64/httpd/modules/mod_pagespeed.so
No symbol table info available.
#2 0x00007ff1348fb8b8 in ap_run_map_to_storage (r=0x7ff1362f0698) at 
/usr/src/debug/httpd-2.2.15/server/request.c:69
pHook = <value optimized out>
n = <value optimized out>
rv = <value optimized out>
#3 0x00007ff1348fd9c8 in ap_process_request_internal (r=0x7ff1362f0698) at 
/usr/src/debug/httpd-2.2.15/server/request.c:150
file_req = 0
access_status = <value optimized out>
#4 0x00007ff13490fa20 in ap_process_request (r=0x7ff1362f0698) at 
/usr/src/debug/httpd-2.2.15/modules/http/http_request.c:280
access_status = <value optimized out>
#5 0x00007ff13490c8f8 in ap_process_http_connection (c=0x7ff1362f46b8) at 
/usr/src/debug/httpd-2.2.15/modules/http/http_core.c:190
r = 0x7ff1362f0698
csd = 0x0
#6 0x00007ff134908608 in ap_run_process_connection (c=0x7ff1362f46b8) at 
/usr/src/debug/httpd-2.2.15/server/connection.c:43
pHook = <value optimized out>
n = <value optimized out>
rv = <value optimized out>
#7 0x00007ff1291272fb in ?? () from /usr/lib64/httpd/modules/mod_spdy.so
No symbol table info available.
#8 0x00007ff129152ccd in ?? () from /usr/lib64/httpd/modules/mod_spdy.so
No symbol table info available.
#9 0x00007ff129152ccd in ?? () from /usr/lib64/httpd/modules/mod_spdy.so
No symbol table info available.
#10 0x00007ff12914f1fb in ?? () from /usr/lib64/httpd/modules/mod_spdy.so
No symbol table info available.
#11 0x00007ff1291328f1 in ?? () from /usr/lib64/httpd/modules/mod_spdy.so
No symbol table info available.
#12 0x00007ff1331ab851 in start_thread (arg=0x7ff12735d700) at 
pthread_create.c:301
__res = <value optimized out>
pd = 0x7ff12735d700
now = <value optimized out>
unwind_buf = {cancel_jmp_buf = {{jmp_buf = {140673721685760, 
-6615953477917025934, 140734441252576, 140673721686464, 4, 7, 
6614986543813548402, 6614960367733758322},
mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, 
cleanup = 0x0, canceltype = 0}}}
not_first_call = <value optimized out>
pagesize_m1 = <value optimized out>
sp = <value optimized out>
freesize = <value optimized out>
#13 0x00007ff132ef911d in clone () at 
../sysdeps/unix/sysv/linux/x86_64/clone.S:115
No locals.

Original issue reported on code.google.com by k...@allnice.be on 23 Jan 2013 at 11:21
@GoogleCodeExporter
Copy link
Author 

I think this might be fixed already in trunk.  Would it be possible for you to 
build from trunk & try it out?

Specifically, special handling was added for reverse-proxies to bypass this 
routine.  It wasn't specifically because r->filename was NULL; we were not 
seeing a crash, but were instead having issues with 403s.  This was Issue 582.

Original comment by jmara...@google.com on 23 Jan 2013 at 12:08

  • Changed state: RequestClarification

@GoogleCodeExporter
Copy link
Author 

Building from trunk: 
https://developers.google.com/speed/docs/mod_pagespeed/build_from_source

Follow the commands for the 'bleeding edge' version.

Original comment by jmara...@google.com on 23 Jan 2013 at 12:14

@GoogleCodeExporter
Copy link
Author 

I think I've build from bleeding edge source and this is what I got:

In response:
  x-mod-pagespeed:1.1.0.0-2410
  x-mod-spdy:0.9.3.3-386

Now I have multiple coredumps per page refresh:
(gdb) bt full
#0  __strrchr_sse42 () at ../sysdeps/x86_64/multiarch/strrchr.S:134
No locals.
#1  0x00007f9c2d08756c in ?? () from /usr/lib64/httpd/modules/mod_pagespeed.so
No symbol table info available.
#2  0x00007f9c380708b8 in ap_run_map_to_storage (r=0x7f9c39c935d8) at 
/usr/src/debug/httpd-2.2.15/server/request.c:69
        pHook = <value optimized out>
        n = <value optimized out>
        rv = <value optimized out>
#3  0x00007f9c380729c8 in ap_process_request_internal (r=0x7f9c39c935d8) at 
/usr/src/debug/httpd-2.2.15/server/request.c:150
        file_req = 0
        access_status = <value optimized out>
#4  0x00007f9c38084a20 in ap_process_request (r=0x7f9c39c935d8) at 
/usr/src/debug/httpd-2.2.15/modules/http/http_request.c:280
        access_status = <value optimized out>
#5  0x00007f9c380818f8 in ap_process_http_connection (c=0x7f9c39c8f5b8) at 
/usr/src/debug/httpd-2.2.15/modules/http/http_core.c:190
        r = 0x7f9c39c935d8
        csd = 0x0
#6  0x00007f9c3807d608 in ap_run_process_connection (c=0x7f9c39c8f5b8) at 
/usr/src/debug/httpd-2.2.15/server/connection.c:43
        pHook = <value optimized out>
        n = <value optimized out>
        rv = <value optimized out>
#7  0x00007f9c2c68d2fb in ?? () from /usr/lib64/httpd/modules/mod_spdy.so
No symbol table info available.
#8  0x00007f9c2c6b8ccd in ?? () from /usr/lib64/httpd/modules/mod_spdy.so
No symbol table info available.
#9  0x00007f9c2c6b8ccd in ?? () from /usr/lib64/httpd/modules/mod_spdy.so
No symbol table info available.
#10 0x00007f9c2c6b51fb in ?? () from /usr/lib64/httpd/modules/mod_spdy.so
No symbol table info available.
#11 0x00007f9c2c6988f1 in ?? () from /usr/lib64/httpd/modules/mod_spdy.so
No symbol table info available.
#12 0x00007f9c36920851 in start_thread (arg=0x7f9c2aaad700) at 
pthread_create.c:301
        __res = <value optimized out>
        pd = 0x7f9c2aaad700
        now = <value optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {140308707464960, -7206163316611682514, 140736396973904, 140308707465664, 4, 7, 7261709088800173870, 7261770178195156782},
              mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <value optimized out>
        pagesize_m1 = <value optimized out>
        sp = <value optimized out>
        freesize = <value optimized out>
#13 0x00007f9c3666e11d in clone () at 
../sysdeps/unix/sysv/linux/x86_64/clone.S:115
No locals.

Original comment by k...@allnice.be on 23 Jan 2013 at 2:32

@GoogleCodeExporter
Copy link
Author 

When you did the 'make' did you do 'make BUILDTYPE=Release' ?

The stack trace still shows no debug so you're using a Release build, but to 
get line numbers you need a Debug build: make BUILDTYPE=Debug

Original comment by matterb...@google.com on 23 Jan 2013 at 2:36

@GoogleCodeExporter
Copy link
Author 

I'm not sure I did everything correct but here is my new coredump:

(gdb) bt full
#0  __strrchr_sse42 () at ../sysdeps/x86_64/multiarch/strrchr.S:134
No locals.
#1  0x00007f0597e71f98 in net_instaweb::instaweb_map_to_storage 
(request=0x7f05a4444a08) at net/instaweb/apache/instaweb_handler.cc:993
        filename_starting_at_last_slash = 0x7f05a4472bf0 "\235\t0\243\005\177"
        ret = -1
#2  0x00007f05a349a8b8 in ap_run_map_to_storage (r=0x7f05a4444a08) at 
/usr/src/debug/httpd-2.2.15/server/request.c:69
        pHook = <value optimized out>
        n = <value optimized out>
        rv = <value optimized out>
#3  0x00007f05a349c9c8 in ap_process_request_internal (r=0x7f05a4444a08) at 
/usr/src/debug/httpd-2.2.15/server/request.c:150
        file_req = 0
        access_status = <value optimized out>
#4  0x00007f05a34aea20 in ap_process_request (r=0x7f05a4444a08) at 
/usr/src/debug/httpd-2.2.15/modules/http/http_request.c:280
        access_status = <value optimized out>
#5  0x00007f05a34ab8f8 in ap_process_http_connection (c=0x7f05a44409e8) at 
/usr/src/debug/httpd-2.2.15/modules/http/http_core.c:190
        r = 0x7f05a4444a08
        csd = 0x0
#6  0x00007f05a34a7608 in ap_run_process_connection (c=0x7f05a44409e8) at 
/usr/src/debug/httpd-2.2.15/server/connection.c:43
        pHook = <value optimized out>
        n = <value optimized out>
        rv = <value optimized out>
#7  0x00007f05971c02fb in ?? () from /usr/lib64/httpd/modules/mod_spdy.so
No symbol table info available.
#8  0x00007f05971ebccd in ?? () from /usr/lib64/httpd/modules/mod_spdy.so
No symbol table info available.
#9  0x00007f05971ebccd in ?? () from /usr/lib64/httpd/modules/mod_spdy.so
No symbol table info available.
#10 0x00007f05971e81fb in ?? () from /usr/lib64/httpd/modules/mod_spdy.so
No symbol table info available.
#11 0x00007f05971cb8f1 in ?? () from /usr/lib64/httpd/modules/mod_spdy.so
No symbol table info available.
#12 0x00007f05a1d4a851 in start_thread (arg=0x7f0594bdf700) at 
pthread_create.c:301
        __res = <value optimized out>
        pd = 0x7f0594bdf700
        now = <value optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {139661947041536, 3703545060135028190, 140735242467712, 139661947042240, 4, 7, -3643821220001987106, -3643727428746983970},
              mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <value optimized out>
        pagesize_m1 = <value optimized out>
        sp = <value optimized out>
        freesize = <value optimized out>
#13 0x00007f05a1a9811d in clone () at 
../sysdeps/unix/sysv/linux/x86_64/clone.S:115
No locals.

Original comment by k...@allnice.be on 23 Jan 2013 at 3:45

@GoogleCodeExporter
Copy link
Author 

OK I think we just need to add the null-check as well.  Evidently WebSphere or 
whatever sets r->filename=NULL but does not set r->proxyreq==PROXYREQ_REVERSE, 
so we just need to change line 960 to:

  if ((request->proxyreq == PROXYREQ_REVERSE) || (request->filename == NULL)) {
...

Would that be easy for you to try?

Original comment by jmara...@google.com on 23 Jan 2013 at 3:48

  • Changed state: Accepted

@GoogleCodeExporter
Copy link
Author 

[deleted comment]

@GoogleCodeExporter
Copy link
Author 

Seem to be working now \o/

Original comment by k...@allnice.be on 23 Jan 2013 at 4:03

@GoogleCodeExporter
Copy link
Author 

Fixed as of r2448.

Original comment by matterb...@google.com on 29 Jan 2013 at 7:51

  • Changed state: Fixed
  • Added labels: Milestone-v25

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
auto-migrated Milestone-v25 Priority-Medium Type-Defect
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@GoogleCodeExporter