My favorites | Sign in
Project Logo
          
New issue | Search
for
| Advanced search | Search tips
Issue 201: Patch: Limit Memory of an ApplicationServer Instance
16 people starred this issue and may be notified of changes. Back to list
Status:  New
Owner:  ----
Type-Defect
Priority-Medium


Sign in to add a comment
 
Reported by schmalowsky, Feb 12, 2009
I'd like to supply a patch to limit the max memory usage of an
ApplicationServer instance.

Example:
If you have set RailsMaxInstancesPerApp to 4 then each instance's memory is
limited to PassengerMemLimit.

PassengerMemLimit can be given in (K|M)Bytes, e.g.

PassengerMemLimit 134217728
or
PassengerMemLimit 131072K
or
PassengerMemLimit 128M


It would be great if this patch finds its way into the official source. And
I hope that other might find it useful, too.


PASSENGER-APPLICATION_SERVER-MEMORY_LIMIT.patch
19.1 KB Download
Comment 1 by nmeans, Apr 10, 2009
This should probably be tagged as "Enhancement" rather than "Defect", since it not being there doesn't really 
cause Passenger to fail in some way.  Would definitely be a useful enhancement.

In the meantime, the "PassengerMaxRequests" setting comes in handy to address this problem.  Just watch your 
Passenger processes and see how many requests it takes them to start getting too big, then set MaxRequests 
lower than that number and Passenger will kill the process off after it has served that number of requests.  It's 
not as ideal as a mem limit would be, but it works for now.
Comment 2 by james....@workbooks.com, Apr 20, 2009
A strong "+1" from me.   I recently had a defect which caused the full 16GB of my server to be occupied by just 
one of my Passenger Rails processes, forcing me to power-cycle it.   I attempted to use "ulimit" which did not 
work.  The patch here appears to be a workable solution and I think would be used by many.  Thanks.


Comment 3 by woody.peterson, Apr 28, 2009
So would this kill the spawn before or after reaching the memory limit? Sure, it
would be handy to have something to prevent your apps from sucking up 16GB of memory,
but it would be REALLY nice to have something that frees the memory of big requests
after they've completed. Yes, max requests works, but seems like a workaround if this
is your goal.
Comment 4 by james....@workbooks.com, Apr 28, 2009
This limit causes the process to die as it attempts to exceed its maximum permitted size; the request would 
fail.   As such it is a last resort (but sometimes things don't work as you would want them to and you need a 
fallback).   

At the end of our request processing logic we have already patched Passenger's abstract_request_handler to 
ensure that all the garbage collection is concentrated in one place: the alternative is to leave it to ruby's 
defaults which often cause multiple garbage collection scans per request: very inefficient.   Before garbage 
collecting we are going to check for large process size and exit when the process is overly large by checking 
the RSS: @Woody, is this what you are referring to?

There is really no substitute for asking the OS to enforce hard limits for when your code is not behaving. If 
you don't and you are on Linux then you are at the mercy of the OOM Killer.  @nmeans, although Passenger's 
code is not defective, the absence of this function does mean that the solution as a whole is less robust than 
it could be.



Comment 5 by l...@lukeludwig.com, Jun 23, 2009
This feature, restarting app instances when their memory grows beyond a max limit,
would be found useful by many people. For example, our app is hosted at Engine Yard
and they have even written a shell script for their customers which we run as a cron
and which uses passenger-memory-stats to determine if an instance has consumed more
than a given memory limit and kills the instance if it has.  This works, but it would
be nice if it was built-in. 
Comment 6 by frankske, Sep 04, 2009
Would be real handy, esp for passenger nginx. The PassengerNginxHelperServer is not a subprocess of nginx, so 
the "normal" monit stuff won't help.
Comment 7 by John.R.Mandel, Oct 23, 2009
This is exactly what we need to defend against errors in the system that cause the passenger instance to grow 
wildly out of control.  Right now on EC2 when you run out of memory the instance freezes so this is a system 
down situation when such a bug (albeit ours) occurs.
Sign in to add a comment

Hosted by Google Code