modwsgi - InstallationOnMacOSX.wiki

summary Installation of mod_wsgi on MacOS X.

labels Phase-Deploy

If you are using mod_wsgi, please consider making a donation.

Installation On MacOS X

If using a MacOS X system, mod_wsgi can be compiled from source code against the standard versions of Python and Apache supplied with MacOS X. The GNU C compiler from the MacOS X Developer Toolkit bundle is required.

For descriptions of general problems that may be encountered during installation on MacOS X, see the documentation on "Installation Issues".

The following MacOS X specific issues can also arise.

Non Standard Python Version

Note: This issue is fixed in mod_wsgi vesion 2.0. It will backported to version 1.4 if such a version needs to be created.

If you wish to use a newer version of Python than what the operating system supplies, it is recommended that you use one of the precompiled installers for Python which are available. Where these installers install the newer version of Python into the standard frameworks directory, so as to coexist with the operating system version, the mod_wsgi build process should still work.

If using a packaging system for MacOS X packages that installs everything in its own directory hierarchy you may need to modify the 'Makefile' generated by the 'configure' script. The modification required will be to add the necessary compiler options to the LDFLAGS variable to indicate to the linker that it should look in the packaging system's library directory for either the Python framework or shared library. For example:

LDFLAGS = -Wl,-F/opt/local/Library/Frameworks ....

Installing Using MacPorts

Note: This issue is fixed in mod_wsgi vesion 2.0. It will backported to version 1.4 if such a version needs to be created.

If mod_wsgi is compiled against version of Python from MacPorts, when restarting Apache an error of the following form will likely be encountered.

httpd: Syntax error on line 415 of /usr/local/apache2/conf/httpd.conf: \ Cannot load /usr/local/apache2/modules/mod_wsgi.so into server: \ Symbol not found: _PyList_Type\n Referenced from: \ /usr/local/apache2/modules/mod_wsgi.so\n Expected in: flat namespace

If otool is run on the 'mod_wsgi.so' file, it will likely be found that the operating system version of Python is being wrongly used:

$ otool -L mod_wsgi.so mod_wsgi.so: /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 88.3.9) /System/Library/Frameworks/Python.framework/Versions/2.3/Python (compatibility version 2.3.0, current version 2.3.5)

To resolve this, the manual change to LDFLAGS as described above would be required.

Alternatively, an actual MacPorts version of mod_wsgi could be created. An example of the MacPorts configuration file can be found attached to ticket #28.

You will need to consult the MacPorts site to work out what you are supposed to do with that file.

Installation On Leopard

Note: This issue is fixed in mod_wsgi vesion 2.0. It will backported to version 1.4 if such a version needs to be created.

If wishing to install mod_wsgi for MacOS X 10.5 (Leopard) and you encounter the following error when starting Apache:

$ apachectl start httpd: Syntax error on line 117 of /private/etc/apache2/httpd.conf: \ Cannot load /usr/libexec/apache2/mod_wsgi.so into server: \ dlopen(/usr/libexec/apache2/mod_wsgi.so, 10): no suitable image found. \ Did find:\n\t/usr/libexec/apache2/mod_wsgi.so: mach-o, but wrong architecture

Modify the generated 'Makefile' in mod_wsgi source code and change the CFLAGS variable to:

CFLAGS = -Wc,"-arch ppc7400" -Wc,"-arch ppc64" -Wc,"-arch i386" -Wc,"-arch x86_64"

Also, insert into LFDLAGS similar options but not escaped by '-Wc,'.

LDFLAGS = -arch ppc7400 -arch ppc64 -arch i386 -arch x86_64 ...

The '...' should be the existing flags for LDFLAGS.

This is required as the compiler is not creating a full fat version of the Apache module for mod_wsgi and when it is loaded into Apache, it can't find correct version for the architecture. This appears to be mainly occuring where you are running with a 64bit CPU (x86_64), as the compiler is only generating a i386 binary.

You can confirm what is being generated by running:

$ file /usr/libexec/apache2/mod_wsgi.so /usr/libexec/apache2/mod_wsgi.so: Mach-O bundle i386

If it only shows i386 architecture you may have problems. You should see:

$ file /usr/libexec/apache2/mod_wsgi.so /usr/libexec/apache2/mod_wsgi.so: Mach-O universal binary with 4 architectures /usr/libexec/apache2/mod_wsgi.so (for architecture ppc7400): Mach-O bundle ppc /usr/libexec/apache2/mod_wsgi.so (for architecture ppc64): Mach-O 64-bit bundle ppc64 /usr/libexec/apache2/mod_wsgi.so (for architecture i386): Mach-O bundle i386 /usr/libexec/apache2/mod_wsgi.so (for architecture x86_64): Mach-O 64-bit bundle

Missing Code For Architecture

When running the operating system supplied version of Apache on MacOS X 10.5 (Leopard) with a 64 bit CPU, it will run in 64 bit mode. As described above, this necessitates that the mod_wsgi module be compiled with the 64 bit architecture included. This requirement also applies to any Python C extension modules as well and the shared libraries they use.

Unfortunately, when using distutils to build Python packages, it only includes code for 32 bit architectures. Further, some packaged Python C extension modules are also not being supplied as fully fat binaries. That is, they do not contain code for all 4 MacOS X architectures. The result is that when Python attempts to load these C extension modules it cannot find code for the current architecture and can fail.

One C extension module for which this is commonly occuring is the prebuilt MySQL extension module for Python. The result of trying to load this module from within a WSGI application running under mod_wsgi on 64 bit Intel Apple hardware is:

ImproperlyConfigured: Error loading MySQLdb module: \ dlopen(/Library/WebServer/.python-eggs/MySQL_python-1.2.2-py2.5-macosx-10.5-i386.egg-tmp/_mysql.so, 2): no suitable image found. \ Did find: /Library/WebServer/.python-eggs/MySQL_python-1.2.2-py2.5-macosx-10.5-i386.egg-tmp/_mysql.so: no matching architecture in universal wrapper

Note that such errors may not always stand out like this, as Python code which imports the C extension module may catch the import failure and remap it into some other error which doesn't show the underlying error message about a missing architecture. For example, the PIL package may only show up the error message:

The _imaging C module is not installed

This is because it interprets the inability to load the _imaging C extension module as meaning the '.so' file is not installed when what actually happened was that the required architecture was missing.

The preferred solution to these problems is to recompile and reinstall the packages from scratch so that they do include code for all architectures.

For Python packages that are installed using a distutils 'setup.py' file, or those which are installed using 'easy_install', then one way of ensuring all architectures are compiled into any C extension modules is to set the ARCHFLAGS environment variable prior to building and/or installing the package. For example:

$ ARCHFLAGS='-arch ppc -arch ppc64 -arch i386 -arch x86_64' $ export ARCHFLAGS $ easy_install cElementTree

This method has even been documented by Apple in the following developer release note.

Whether this will work may depend on whether the package has in some way itself tried to work out what architectures it should be compiled for. In other words, if the package has already overridden what architectures it should be built for, this may not work.

Thinning The Apache Executable

If recompiling C extension modules to incorporate all architectures is not possible, then the only known work around is to thin the binary for the Apache 'httpd' executable so as to only include code for the 32 bit Intel platform. The lack of the code for other architectures will force the process to run in 32 bit mode rather than 64 bit mode.

To thin the Apache 'httpd' executable one should perform the following steps. Note that changes are being made to operating system supplied files and so backups should be made first.

``` cd /usr/sbin

sudo mv ./httpd ./httpd.fat sudo lipo ./httpd.fat -thin i386 -output ./httpd.i386 sudo ln -s ./httpd.i386 ./httpd ```

After having done this, you should end up with:

``` $ file ./httpd.fat ./httpd.fat: Mach-O universal binary with 4 architectures ./httpd.fat (for architecture ppc7400): Mach-O executable ppc ./httpd.fat (for architecture ppc64): Mach-O 64-bit executable ppc64 ./httpd.fat (for architecture i386): Mach-O executable i386 ./httpd.fat (for architecture x86_64): Mach-O 64-bit executable x86_64

$ file ./httpd.i386 ./httpd.i386: Mach-O executable i386 ```

When Apache is now run, it should run as Intel and not Intel (64 bit). You can check what it runs as using the 'Activity Monitor' application. Look at the 'Kind' column to see what it runs as.

It should then be possible to load C extension modules into Python which only provide code for 32 bit Intel architecture.

Note that if you have applied any operating system updates which replace the 'httpd' executable, you will need to perform the above steps again.

According to the manual page for 'arch', as an alternative to thining the executable, it may be possible to define the environment variable 'ARCHPREFERENCE' appropriately in the Apache 'envvars' file located in the same directory as the 'httpd' executable. So far experiments in using this approach have failed though to force Apache to run in 32 bit Intel mode in preferences to 64 bit Intel mode. Experiments running 'httpd' using the 'arch' command directly from the 'apachectl' script to force it to run in 32 bit Intel mode have also failed.

Forcing 32 Bit Execution

Instead of actually thining the Apache executable so as to only include the code for the 32 bit architectures, it is also possible to modify the 'plist' file which controls the startup of Apache. This file is located at:

/System/Library/LaunchDaemons/org.apache.httpd.plist

Because this is a system file, it is though preferable that it not be modified directly. Instead, copy the file to:

/Library/LaunchDaemons/org.apache.httpd.plist

Then edit the file and for Intel architecture change:

<string>/usr/sbin/httpd</string> <string>-D</string> <string>FOREGROUND</string>

to:

<string>/usr/bin/arch</string> <string>-i386</string> <string>/usr/sbin/httpd</string> <string>-D</string> <string>FOREGROUND</string>

This will have the effect of launching Apache via the 'arch' command with the '-i386' option supplied to 'arch' to force Apache to run in 32 bit mode.

/usr/bin/arch -arch i386 /usr/sbin/httpd -D FOREGROUND

If running a 64 bit PowerPC machine then use the appropriate option for that platform instead. Check the manual page for 'arch' for more details.

To activate the configuration, reboot the system or run:

sudo launchctl unload /System/Library/LaunchDaemons/org.apache.httpd.plist sudo launchctl load /Library/LaunchDaemons/org.apache.httpd.plist

Note that after upgrades of the operating system you may wish to validate that no changes were made to the original file and if necessary copy the 'plist' file again and reapply changes.

Non Universal Developer Tools

When 'configure' is run for mod_wsgi, it will work out what architectures the version of MacOS X being used may support by examining what architectures are compiled into the version of Apache being used. For example:

$ file /usr/sbin/httpd /usr/sbin/httpd: Mach-O universal binary with 4 architectures /usr/sbin/httpd (for architecture ppc7400): Mach-O executable ppc /usr/sbin/httpd (for architecture ppc64): Mach-O 64-bit executable ppc64 /usr/sbin/httpd (for architecture i386): Mach-O executable i386 /usr/sbin/httpd (for architecture x86_64): Mach-O 64-bit executable x86_64

With MacOS X 10.5 (Leopard) four different architectures are capable of being supported by binaries. These are 'ppc7400', 'ppc64', 'i386' and 'x86_64', and these all exist within the Apache executable. With older versions of MacOS X, Apache would typically only be compiled for the single architecture that the operating system is being run on. Whatever the case, the build scripts for mod_wsgi will tell the GNU compiler being used to generate a binary containing all architectures which are supported by the version of Apache being used.

For MacOS X 10.5 (Leopard), the GNU compilers provided with the operating system are capable of producing fat binaries with all architectures, and the version of Python supplied with the operating system is also compatible with being used under all architectures. If however you installed MacPorts versions of the GNU C compilers and of Python, especially if you inherited those tools from an upgrade of Tiger to Leopard, you may run into problems with the tools not being able to produce fat binaries for all architectures.

One way this may manifest is in the form of the compile time error:

error: #error "LONG_BIT definition appears wrong for platform (bad gcc/glibc config?)."

If this occurs, you would ideally upgrade your MacPorts versions of the GNU compiler and Python to more recent versions which are capable of being used to produce fat binaries for multiple architectures. If you cannot do this, then you should edit the 'Makefile' generated by the mod_wsgi 'configure' script and remove references to additional architectures, leaving only that for your own behind.

CFLAGS = -Wc,'-arch ppc7400' LDFLAGS = -arch ppc7400 ...

The '...' should be the existing flags in LDFLAGS following the architecture flags.

Note that the error:

error: #error "LONG_BIT definition appears wrong for platform (bad gcc/glibc config?)."

may also be due to the version of the Python framework being used not having being configured and compiled with support for 64 bit architectures.

Non Universal MacPython

The Python Software Foundation provides a precompiled Python distribution for MacOS X referred to as MacPython. The Python framework this contains does not however support 64 bit architectures. This means that although mod_wsgi may be configured and built for 64 bit architectures where Apache being used supports it, a run time failure will occur when Apache is started.

The error encountered will be of the type:

If you want to use a version of Python newer than that provided by the MacOS X operating system and do not wish to use MacPorts Python, then you will need to compile Python from source code. In compiling Python from source code, it is essential you do what the MacPython folks didn't, which is to enable the generation of a universal framework containing all architectures.

The options required to the 'configure' script for Python on MacOS X (10.5) to ensure that all architectures are generated is:

./configure --prefix=/usr/local/python-2.6.4 \ --enable-framework=/usr/local/python-2.6.4/frameworks \ --enable-universalsdk=/ MACOSX_DEPLOYMENT_TARGET=10.5 \ --with-universal-archs=all

The options required to the 'configure' script for Python 2.6+ or Python 3.2+ on MacOS X (10.6) to ensure that all supported architectures are generated is:

./configure --prefix=/usr/local/python-2.6.4 \ --enable-framework=/usr/local/python-2.6.4/frameworks \ --enable-universalsdk=/ MACOSX_DEPLOYMENT_TARGET=10.5 \ --with-universal-archs=3-way

Take note how with Snow Leopard you must use '3-way' instead of 'all'.

It is recommended that self compiled Python versions be installed into their own directory rather than allowing default '/Library' directory be used.