I am using mod-wsgi with django, and in django I use pylucene to do full text search.

While mod-wsgi is configured to be embedded mode, there is no problem at all. But when mod-wsgi is configured to be daemon mode, the apache just gets stuck, and the browser just keep loading but nothing appears.

Then I identity the problem to be the jcc.initVM(). Here is my wsgi script:

import os, sys, jcc
sys.stderr.write('jcc.initVM\n')
jcc.initVM()
sys.stderr.write('finished jcc.initVM\n')
....

After I restart my apache, and make a request from my browser, I find that /var/log/apache2/error.log only has:

jcc.initVM

Meaning that it gets stuck at the line jcc.initVM(). (If the mod_wsgi is configured as embedded mode, there is no problem.)

And here is my /etc/apache2/sites-available/default:

WSGIDaemonProcess site user=ross group=ross threads=1
WSGIProcessGroup site
WSGIScriptAlias / /home/ross/apache/django.wsgi

<Directory /home/ross/apache/>
  Order deny,allow
  Allow from all
</Directory>

And finally, I find out that in the source code of jcc (jcc.cpp), it hangs at the function:

JNI_CreateJavaVM(&vm, (void **) &vm_env, &vm_args)

How to solve the problem?

Program versions:

libapache2-mod-wsgi 2.3-1
jcc 2.1
python 2.5
Apache 2.2.9-8ubuntu3
Ubuntu 8.10

Probably same as:

http://code.google.com/p/modwsgi/issues/detail?id=87

No simple solution yet except use a wrapper program around your program that unblocks signals.

I am sorry that I still don't know how to work around.

django.wsgi: import os os.system('/home/ross/apache/run.py')

run.py:

!/usr/bin/env python

import os, sys import jcc sys.stderr.write('jcc.initVM\n') jcc.initVM() sys.stderr.write('finished jcc.initVM\n') sys.path.append('/home/ross/voofie/') sys.path.append('/home/ross/') os.environ['DJANGO_SETTINGS_MODULE'] = 'voofie.settings' import django.core.handlers.wsgi application = django.core.handlers.wsgi.WSGIHandler()

It won't work. Could you give me some suggestion?

It isn't that simple unfortunately. The wrapper program needs to be a C program as Python doesn't have access to required functions. Specifically, calls need to be made to either of the C functions sigblock() or sigprocmask() to restore ability of the thread that exec'd the program to receive signals.

I'll have another look at whether there might be a fiddle that can be done in mod_wsgi to allow it to work, but don't think there is. The code in mod_wsgi has to be written a completely different way and I just haven't got to that yet.

Thank you very much.

With reference to Brennan's exec.c, I write the following code using cython and compiled that.

cpyrun.pyx:

cdef extern from "signal.h": ctypedef struct sigset_t: pass int sigfillset(sigset_t *set) int sigprocmask(int __how, sigset_t *set, sigset_t *oset) int SIG_UNBLOCK

import os, sys import jcc from django.core.handlers.wsgi import WSGIHandler def run_wsgi(): cdef sigset_t set sigfillset(&set) sigprocmask(SIG_UNBLOCK, &set, NULL) sys.stderr.write('jcc.initVM\n') jcc.initVM() sys.stderr.write('finished jcc.initVM\n') sys.path.append('/home/ross/site/') sys.path.append('/home/ross/') os.environ['DJANGO_SETTINGS_MODULE'] = 'site.settings' application = WSGIHandler()

And my django.wsgi:

import sys sys.path.append('/home/ross/site/apache') import cpyrun cpyrun.run_wsgi()

However, I still get the same error, that the program still can't pass jcc.initVM. Does that mean that I used the sigprocmask() wrongly? (I don't have much idea about that, I just copy that from the exec.c and change that to cython)

I'd still have:

django.wsgi: import os os.system('/home/ross/apache/run.py')

run.py:

!/usr/bin/env python

...

and have signal unblocking in separate run.py program rather than in mod_wsgi daemon process as signals still going to likely be captured by Apache/mod_wsgi and not end up being handled by your program.

This is all assuming it is the same problem with SIGPWR.

Sorry, I don't get what you mean in your comment 5. In comment 3, you said I have to use sigblock() or sigprocmask() to unblock signals, and I did that with comment 4's code. However it still didn't work out. In comment 5, you referred back to the code in comment 2, which you already said won't work since I didn't use sigblock() or sigprocmask(). Could you have a look to comment 4's code, and give me some feedback if the code use sigprocmask() correctly? (Since I am not good in this.) Thank you very much.

The point I was making was that you moved the unblocking of signals back into the WSGI script. Ie., you have WSGI script that runs inside of Apache process importing the module directly that tries to unblock signals. This possibly will not work. Your WSGI script file still has to use os.system() to execute a separate external script, with that external script containing the unblocking and then doing the Java stuff. I don't know how the C wrapping tool you are using works. But something to the effect of:

django.wsgi: import os os.system('/home/ross/apache/run.py')

run.py:

!/usr/bin/env python

import sys import jcc

import cpyrun cdef sigset_t set cpyrun.sigfillset(&set) cpyrun.sigprocmask(SIG_UNBLOCK, &set, NULL)

sys.stderr.write('jcc.initVM\n') jcc.initVM() sys.stderr.write('finished jcc.initVM\n') ..... other stuff you need to do

if I do that: django.wsgi: import os os.system('/home/ross/apache/run.py')

It complains that: [error] [client 127.0.0.1] mod_wsgi (pid=5687): Target WSGI script '/home/ross/apache/django.wsgi' does notcontain WSGI application 'application'.

I don't mean literally that. I only copied that from what you had said in comment 2. I took that only as indication of what you were going to have your Django application do, it may not even be in the WSGI script file but in some code associated with your function which is triggered by the URL when requested.

If it was in the WSGI script file, then you are perhaps going about this all the wrong way anyway. Remember that Apache/mod_wsgi can run as multiprocess configuration. The last thing you possibly want is this stuff from from inside of the web server processes anyway, especially if having multiple copies running is a bad thing.

Thus, you migher be better off having a standalone process that is executed distinct from the actual Django instance, that sets environment variables, imports Django infrastructure then does what it needs to. That is, it would actually serve requests.

Anyway, I am guess here as have no idea how pylucene works or what it really does. Since it mentions a JNI function I am just assuming it is Java stuff, which as per previous report doesn't work properly because of signals being blocked.

The question now is, does pylucence have to be in the web server processes. Can it just run in its own process with it perhaps populating database with index information which is then access via the web server processes?

I believe I have a suitable fix for the underlying signal problem, presuming that is what is affecting you.

I had been holding off on this issue as felt I needed to first fix:

http://code.google.com/p/modwsgi/issues/detail?id=102

which touches related code, so wanted to avoid having to make two sets of changes.

I'll look at doing the interim change for the signal problem first though even if have to change it again later.

Thank you very much for your quick response and useful suggestion. I think I will move the searching code to a totally new program let it communicate with the program in mod_wsgi.

One thing forgot to mention to try in case this is nothing to do with signals. Deletegate WSGI application to run in main Python interpreter.

WSGIApplicationGroup %{GLOBAL}

Some third party extension modules for Python use simplified GIL API and will only work in main interpreter.

See:

http://code.google.com/p/modwsgi/wiki/ApplicationIssues#Python_Simplified_GIL_State_API

I have implemented fix for avoiding signal problem, but have to still validate it and commit it back to trunk.

Marking this as duplicate, specifically of:

http://code.google.com/p/modwsgi/issues/detail?id=87

until can be determined otherwise. The signal issue has now been addressed.

This issue may also have been connected with code simply not being run in main interpreter.