Using Python with Nginx via Passenger

Since Nginx 0.6.* I been looking for an effective way to run Python (WSGI) applications thru the Nginx Webserver. A couple of options were available at the time:

Update 1/31/2011: There is now an article showing how to deploy Python (namely with Circuits.web) via Nginx + uWSGI.
Deploying circuits.web with Nginx/uwsgi.

Update 9/29/2010: As of version 0.8.40 Nginx now supports wsgi applications running over the uWSGI protocol natively. Click Here for more details. The 0.8.x branch is now the current stable branch.

Apache + mod_wsgi
One possible solution was to run Apache as a backend server listening on 127.0.0.1:8080 (or a port of your choice), which could also be easily used to serve up other dynamic content such as php. Many have used this setup for a great deal of internal load balancing, letting Nginx serve strictly static content, and Apache to handle the dynamic content.

In the sense of a WSGI Python application, you would have a server block like this in Nginx:

server {
	#server_name is what nginx responds to
	server_name  mydomain.com www.mydomain.com;
 
	location / {
		#setting a Host header is handy if you have apache
		#setup with name-based virtual hosts
		proxy_set_header Host mydomain.com;
 
		#requires nginx to be compiled with --with-http_realip_module
		proxy_set_header X-Real-IP $remote_addr;
 
		#This can also be the IP address of another server
		proxy_pass http://127.0.0.1:8080;
	}
}

With the above method you could go with the easy route of having mod_wsgi (as of this writing, mod_wsgi is available at 2.5 supporting Python 2.6) installed via Apache. However doubling up the number of webservers run, might defeat the purpose of running Nginx for some of us, especially for those less familiar with Apache configuration.

Nginx WSGI Module

There was a port of Apache’s mod_wsgi over to Nginx by Manlio Perillo. However this module had only been tested on Nginx 0.5.34, and patched for 0.6.*.

However even when patched, the module remained quite buggy. Implementing this module went something like this:

In the http { } block

    wsgi_python_optimize 0;
    wsgi_enable_subinterpreters on;

In your server { } block

server {
	listen 80;
	server_name yourdomain.com;
 
        location / {
            wsgi_pass  /var/www/username/domain/application.wsgi;
 
            include wsgi_vars;
 
            wsgi_var  SCRIPT_NAME         /var/www/username/domain/application.wsgi;
            wsgi_var  DOCUMENT_ROOT       /var/www/username/domain;
 
            wsgi_pass_authorization off;
            wsgi_script_reloading on;
            wsgi_use_main_interpreter on;
        }
}

This module is no longer maintained and is does not to build on modern versions of Nginx (0.7/0.8), and was barely compatible with 0.6 by using a patch. The creator of the Apache mod_wsgi, Graham Dumpleton commented on his blog regarding the Nginx implementation:


The nginx version of mod_wsgi borrows some code from my original Apache version, but obviously since the internals of Apache and nginx are very different, the main parts of the code which interface with the web server are unique. Although I condoned use of the source code, I do wish I had insisted from the outset that it not be called mod_wsgi due to the confusion that has at times arisen.

Although development on the nginx version of mod_wsgi appears to no longer be happening, this isn’t stopping people from using it and many are quite happy with it. The question is whether they really understand anything about how nginx works and the shortcomings in how nginx and mod_wsgi work together.

Admittedly the author of the mod_wsgi module for nginx has been up front in pointing out that because nginx is asynchronous, and with WSGI not designed for such a system, that once the WSGI application is entered all other activity by the web server is blocked. The recommendation resulting from this is that static files should not be served from the same web server. Use of multiple nginx worker processes is also suggested as a way of mitigating the problem.

Source: Graham Dumpleton: Blocking requests and nginx version of mod_wsgi

There is however a far more practical solution fit for production use, especially for those using modern versions of Nginx.

5 comments

  1. Hongli Lai says:

    Hi kbeezie, good writeup. :) I just published a blog article about Phusion Passenger and Python today. Maybe you’d care to comment? http://izumi.plan99.net/blog/index.php/2009/11/21/phusion-passenger-for-python/

    “We’ll need to download and compile Ruby version 1.9.1 patch level 129. While this is not the latest, I’ve had unconfirmed reports that anything newer than p129 is not fully compatible with Passenger.”

    This is correct. Later versions of 1.9.1 have bugs in the ‘tempfile’ library. These bugs have been fixed in Ruby’s SVN repository but so far there hasn’t been a 1.9.1 release containing the bug fixes.

    By the way, it’s not necessary to install Rails in order to use Phusion Passenger. Rails is only required if you deploy Rails apps.

  2. vanderkerkoff says:

    kbeezie, form some reason sudo is not finding ruby

    any ideas?

  3. kbeezie says:

    Without using sudo, try executing ruby -v to see if anything shows up at all, if not try ‘which ruby’. If nothing still shows up then either the PATH isn’t configured correctly to Ruby, or Ruby itself isn’t properly installed. ‘sudo’ itself doesn’t find commands per se, but is simply a way of saying ‘I want to run the following command as a super-user or root’.

  4. Jay States says:

    Hrm? Why, I’ll assuming that you’ve not seen the uwsgi project? This is much better for python/django/pylons – http://projects.unbit.it/uwsgi/

    Good Luck

  5. kbeezie says:

    Note the date of Nginx’s support for uwsgi and the date of said article.