For the purpose of this write up, I am demonstrating the configuration as they are on a FreeBSD 9.0 server. The ‘www’ user and group are often referred to as ‘www-data’ on most linux hosts. For saftey Nginx and PHP-FPM are run as www:www, with ownership of the web files belonging to an unprivileged user and permissions set accordingly.

On most of my servers my web files are owned by an unprivileged user that belongs to the www group, and I tend to run nginx and php as www:www. Running PHP as said user:www can be quite risky since most files have read/write/execute permission set on the owner bit. (in octal permissions it’s UGO – User/Owner-Group-Other, such as 750 = owner(read/write/excute) + group (read/execute) + other (none))

In this write up I am simply going to show you the configuration of second a second smaller PHP-FPM pool to act as an ‘elevated’ process running as the folder’s owner, thus allowing far more control over the files and folders such as upgrading wordpress or it’s plugins, but not allow any other files not matching the pattern to do the same.

First thing we need to do is add a second pool to the php-fpm.conf, on FreeBSD installed from ports it will be located at /usr/local/etc/php-fpm.conf, on other systems such as debian or ubuntu, it may be located as /etc/php5/fpm/pool.d/default or similar (look for the config with the [www] pool in it).

This configuration has been stripped down quite a bit with some commented options shown for example purposes.

[global]
pid = run/php-fpm.pid
 
[www]
user = www
group = www
listen = /tmp/php.sock
listen.owner = www
listen.group = www
listen.mode = 0666
 
;listen is not required with sockets
;listen.allowed_clients = 127.0.0.1
 
pm = static 
pm.max_children = 4
pm.start_servers = 2
pm.min_spare_servers = 1
pm.max_spare_servers = 3
pm.max_requests = 1024 
 
;chroot = 
;chdir = /var/www
;catch_workers_output = yes
;security.limit_extensions = .php .php3 .php4 .php5
 
;env[HOSTNAME] = $HOSTNAME
;env[PATH] = /usr/local/bin:/usr/bin:/bin
;env[TMP] = /tmp
;env[TMPDIR] = /tmp
;env[TEMP] = /tmp
 
;php_admin_value[sendmail_path] = /usr/sbin/sendmail -t -i -f www@my.domain.com
;php_flag[display_errors] = off
;php_admin_value[error_log] = /var/log/fpm-php.www.log
;php_admin_flag[log_errors] = on
;php_admin_value[memory_limit] = 32M
 
[elevated]
user = kbeezie
group = www
listen = /tmp/php-elevated.sock
listen.owner = www
listen.group = www
listen.mode = 0666
;listen.allowed_clients = 127.0.0.1
pm = static 
pm.max_children = 2
pm.start_servers = 2
pm.min_spare_servers = 1
pm.max_spare_servers = 3
pm.max_requests = 1024

In the above I have two pools set up, the first one being a typical 4-process (with static, min/max_*servers values are ignored) running as www:www, followed by a second pool named elevated running with 2 processes with the user/group of kbeezie:www (the web files are owned by kbeezie:www).

You’ll notice from some of the commented values above, that you can tweak specific configurations for each pools as needed, such as needing more memory or being locked into a specific folder.

Now in a very simple server block on Nginx for wordpress we need to add a couple location blocks using a regular expression location (this way it overrides the usual php location block when placed ahead of it).

	server { 
		listen 80;
		server_name mywebsite.com;
		root /home/kbeezie/www/mywebsite.com;
 
		location / {
			try_files $uri $uri/ /index.php;
		}
 
		# matches for update.php or update-core.php in /wp-admin/
		location ~ ^/wp-admin/(?:update|update-core)\.php$ {
			include php-elevated.conf;
		}
 
		include php.conf;
		include drop.conf;
	}

I tend to use config files for my PHP inclusion. For example here are both the php.conf and php-elevated.conf located in the same folder as nginx.conf:

location ~ \.php$ {
        try_files $uri =404;
 
        fastcgi_param  QUERY_STRING       $query_string;
        fastcgi_param  REQUEST_METHOD     $request_method;
        fastcgi_param  CONTENT_TYPE       $content_type;
        fastcgi_param  CONTENT_LENGTH     $content_length;
 
        fastcgi_param  SCRIPT_NAME        $fastcgi_script_name;
        fastcgi_param  SCRIPT_FILENAME    $request_filename;
        fastcgi_param  REQUEST_URI        $request_uri;
        fastcgi_param  DOCUMENT_URI       $document_uri;
        fastcgi_param  DOCUMENT_ROOT      $document_root;
        fastcgi_param  SERVER_PROTOCOL    $server_protocol;
 
        fastcgi_param  GATEWAY_INTERFACE  CGI/1.1;
        fastcgi_param  SERVER_SOFTWARE    nginx;
 
        fastcgi_param  REMOTE_ADDR        $remote_addr;
        fastcgi_param  REMOTE_PORT        $remote_port;
        fastcgi_param  SERVER_ADDR        $server_addr;
        fastcgi_param  SERVER_PORT        $server_port;
        fastcgi_param  SERVER_NAME        $server_name;
 
	fastcgi_pass unix:/tmp/php.sock;
}

location ~ \.php$ {
        try_files $uri =404;
 
        fastcgi_param  QUERY_STRING       $query_string;
        fastcgi_param  REQUEST_METHOD     $request_method;
        fastcgi_param  CONTENT_TYPE       $content_type;
        fastcgi_param  CONTENT_LENGTH     $content_length;
 
        fastcgi_param  SCRIPT_NAME        $fastcgi_script_name;
        fastcgi_param  SCRIPT_FILENAME    $request_filename;
        fastcgi_param  REQUEST_URI        $request_uri;
        fastcgi_param  DOCUMENT_URI       $document_uri;
        fastcgi_param  DOCUMENT_ROOT      $document_root;
        fastcgi_param  SERVER_PROTOCOL    $server_protocol;
 
        fastcgi_param  GATEWAY_INTERFACE  CGI/1.1;
        fastcgi_param  SERVER_SOFTWARE    nginx;
 
        fastcgi_param  REMOTE_ADDR        $remote_addr;
        fastcgi_param  REMOTE_PORT        $remote_port;
        fastcgi_param  SERVER_ADDR        $server_addr;
        fastcgi_param  SERVER_PORT        $server_port;
        fastcgi_param  SERVER_NAME        $server_name;
 
	fastcgi_pass unix:/tmp/php-elevated.sock;
}

The only difference you’ll notice is that php-elevated.conf connects to the elevated socket rather than the usual one.

Now once we’ve restarted Nginx and PHP-FPM with the new configuration, when you attempt to update wordpress or one of it’s plugins, the script will have an elevated permission due to running as the files/folder’s owner, while any other scripts (as called by the browser) will run with unprivileged www:www status.

The drop.conf is simply a number of locations that should normally be dropped from public requests, I paste it here for reference:

	# the first line can normally be omitted for wordpress, as robots.txt is normally dynamic
	location = /robots.txt  { access_log off; log_not_found off; }
	location = /favicon.ico { access_log off; log_not_found off; }	
	location ~ /\.          { access_log off; log_not_found off; deny all; }
	location ~ ~$           { access_log off; log_not_found off; deny all; }

Cavets

Bear in mind the above configuration are for example purposes, there are quite a few smaller things to keep in mind such as hardening your configuration for more security concerns or optimizing it for performance. For some examples of typical Nginx configurations see : Nginx Configuration Examples

This guide assumes you already have a jail installed, I personally use ezjail-admin. Here are three guides regarding the installation of ezjail-admin. Cyberciti.biz, Erdgeist.org [creator of ezjail-admin], Secure-Computing.net.

The Scenario

You have a FreeBSD VPS with a single IP and you wish to create a FreeBSD jail for additional security and/or isolation. For this write up I’ll illustrate how you can use a single VPS with a jail create on an internal IP with both NAT access and port-forwarding to the jail for specific ports (web, ssh, etc).

Creating the local interface

In your rc.conf we’ll clone the loopback interface to lo1 so that we can use the 192.168.*, 10.*, or 172.16.* for our private jail network.

cloned_interfaces="lo1"
ipv4_addrs_lo1="192.168.0.1-9/29"

The above will create a lo1 loopback device with 192.168.0.1 thru 192.168.0.9 created on that interface. From here we’ll create a jail with 192.168.0.2. Then we’ll configure PF to allow outbound traffic (NAT) from those local addresses as well as pass web (80) and SSH port to a specific jail IP.

Make sure to change the external interface if it is not em0, but rather something like re0.

IP_PUB="Your Public IP Address Here"
IP_JAIL="192.168.0.2"
NET_JAIL="192.168.0.0/24"
PORT_JAIL="{80,443,2020}"
scrub in all
nat pass on em0 from $NET_JAIL to any -> $IP_PUB
rdr pass on em0 proto tcp from any to $IP_PUB port $PORT_WWW -> $IP_JAIL

Any Jail tied to the 192.168.* IP range will have outbound connectivity (needed for installing ports, updates or other packages), but for a specific jail such as your webserver we’ll pass inbound traffic on port 80/443 for the webserver, and 2020 for SSH. You will need to make sure to configure /etc/ssh/sshd_config to listen on 192.168.0.2, Port 2020 or a port of your choosing.

This way you can use the same IP to connect both to the main VPS on one SSH port, but connect to the jail directly on another port.

Once the above are saved in /etc/pf.conf, we need to test it and then start up pf (or restart).

pfctl -nf /etc/pf.conf
service pf start

We can verify the rules with the following command:

# pfctl -sn
nat pass on em0 inet from 192.168.0.0/24 to any -> YOUR_PUBLIC_IP
rdr pass on em0 inet proto tcp from any to YOUR_PUBLIC_IP port = http -> 192.168.0.2
rdr pass on em0 inet proto tcp from any to YOUR_PUBLIC_IP port = https -> 192.168.0.2
rdr pass on em0 inet proto tcp from any to YOUR_PUBLIC_IP port = 2264 -> 192.168.0.2

If you need to perform specific tasks from inside the jail such as ping, traceroute, sockstat and so forth, add the following to the main node’s /etc/sysctl.conf

security.jail.allow_raw_sockets=1

There you have it, once those rules are set up, all the private jails will have outbound traffic allowing you to install ports and download files from other servers, but will also allow specific ports to be forwarded to specific jails (such as forwarding 3306 to a separate database jail from outside of the VPS).

You can for all intents and purposes, forgo the port forwarding above in the pf.conf section, and simply create for example an IPv6-only Jail that still has the ability to to install packages and ports via the outbound NAT traffic thru the single IPv4 address, as not all ports provide an IPv6 compatible download link.

A space saving tip

If you would rather not have to update the ezjail port tree every time there’s an update, you can instead clone a read-only copy of the ports tree from your master node to each of your jails.

First we must shutdown the jail, and remove the symlinked ports folder from the jail, then create a blank folder to be mounted upon.

# cd /usr/jails/myjail.com/usr/
# rm -Rf ./ports
# mkdir ports

Then we’ll edit the jails’ fstab to include the master port tree located at /etc/fstab.myjail_com

/usr/jails/basejail /usr/jails/myjail.com/basejail nullfs ro 0 0
/usr/ports /usr/jails/myjail.com/usr/ports nullfs ro 0 0

Now to make sure that the jail will be able to download distfiles and create a working directory, we need to start up the jail, log into it, and then edit the /etc/make.conf

WRKDIRPREFIX=		/var/ports
DISTDIR=		/var/ports/distfiles
PACKAGES=		/var/ports/packages
INDEXDIR=		/var/ports

Make sure the /var/ports directory exists if it does not already. Once you have those placed in the /etc/make.conf you’ll be able to:

1) Update your ports tree from a single location (the master node)
2) Run ports updates and installation from within the jail without having to restart the jail, or to re-run ezjail-admin update -p.