Thanks for the report.  The symptom of 20k open connections is interesting.

The reason that it's "looping back" to itself is that this is (currently) how 
mod_pagespeed rewrites resources (css images javascript) -- by initiating an 
HTTP fetch using the "serf" package.  When this works, it works well.  But in 
the version of code you have (the last binary that was shipped) the failure 
mode is not good.

We are working on the problem.  We believe the current 'trunk' in SVN solves 
the symptom you are seeing, which is that failed fetches retry.  In your case, 
with 20k *open* connections it looks like your fetches are not failing; they 
are hanging.  In either case, there this will not happen with the latest 
source, or with a binary we will distribute soon, because we've added locking 
to prevent the same URL from being fetched more than once at a time.

But we still want to eliminate the root cause.  Can you share with us more 
about your networking setup?   If you login to a shell on the machine running  
HTTPD, can you 'wget' all the resources reference in the HTML file using the 
full URL of each resource?  That is what mod_pagespeed is trying to do.  In 
setups where HTTPD rejects fetches from localhost or the URLs for the resources 
are not valid from the HTTPD host, this failure might be possible.  Can you 
give further details on this aspect of your system?

We have also run into a certain amount of trouble interfacing to some sites 
with WP plugins, but I'm not sure if this is related to your issue or not.

We should use this bug to track addressing the symptom of requests hanging 
indefinitely.  In particular, we need to very that our usage of the serf 
fetcher that we only allow requests to remain outstanding for a bounded amount 
of time -- when that time expires we should cancel the request and report a 
failure so that we 'remember' the fetch failure in the HTTP cache and avoid 
retrying for 5 minutes.

We'll still need help to identify why these fetches are hanging on 
www.chat-sohbet.com.

Summary was: Server load above 90 !! Apache extremly overloaded, 20k connection 
on port 80 and high usage of processor

I adjusted the summary because that describes a symptom which we believe will 
be resolved with changes made in the latest binary.  However the root cause is 
still not resolved and thus we leave this bug open.

It looks like there's a weird situation in which we can end up rewriting our 
own fetches (in the process spawning new fetches which themselves get 
rewritten...).  I don't really understand how that would happen -- maybe you 
have a link to a bad resource, and apache is configured to redirect 404s back 
to /index.html?

Anyway, I'll add some code to prevent mod_pagespeed from ever rewriting its own 
requests.

Fixed in revision 238.

Issue 46 has been merged into this issue.

Issue 110 has been merged into this issue.

Hi,

Yes, able to get the files with wget... that is the whole problem as at first I 
was tracking the errors but didn't see anything there. Then when I tracked the 
normal connections I saw A LOT of the same trying to connect to the same page.

If you want, I can check my LOG files and give some ? There you can see that it 
get's in a loop and requesting the same file over and over again...

Let me know.

Thanks.

aLeMiM2008: have you upgraded to the latest release, 0.9.10.1-250?  We believe 
we have resolved this infinite recursion issue.