Archive for the ‘Apache’ category

Adding Google Sitemap CPanel Firewall Exception

August 5th, 2014

If you’ve had any experience with installing Google Sitemap Generator on a Cpanel/WHM based server then you may have run into an issue where port 8181 is not accessible. Even if you modify iptables rules, it seems to go back to blocked after a while.

If your WHM setup is utilizing CSF, then you may need to add the exception via WHM.

Locate the CSF configuration section, and then click on Firewall Configuration.

CSF 1st Step

Then find the IP4 (and IPv6 settings if needed) settings, and add 8181 to the list of ports allowed in for TCP Inbound.

CSF Step 2

Once done, you can save and restart CSF. From then on, you should be able to access your Google Sitemap control panel on port 8181.

If you still cannot, due to not utilizing an SSL certificate, and still wish to connect without installing a certificate, you can do so with the following command as root:

/usr/local/google-sitemap-generator/bin/sitemap-daemon remote_admin enable

IPv6 with OpenVz (venet)

February 16th, 2011

This article assumes that the physical node that is setup with OpenVz already has Native IPv6 setup and running, and that you are using OpenVz containers connected with a venet device. For those who need help getting their physical node setup with IPv6 on CentOS/RHEL click here.

Ideally IPv6 works best over a veth adapter, as they have a MAC address and play much more nicely with auto configuration within the container. However not all of us have the luxury of getting an OpenVz container with a bridged veth adapter AND IPv6.

In a nutshell, using a venet adapter suffers the following:

venet devices are not fully IPv6 compliant, but still works if you statically assign IPv6 addresses. They do not properly support MAC addresses and consequently link local addresses and can not play nice with neighbor discovery or router advertisements, router discovery, or auto-conf. They also require additional modifications to the layer 3 forwarding behaviour of the host via sysctl.

The wiki of course tells you how to add an IPv6 address to a container, but doesn’t go into details on actually getting it to work. I only know of these steps to work on a CentOS and Debian container.

It’s important that the following has been set on the physical server for the following guide to work

sysctl net.ipv6.bindv6only 1

Adding an IP

In the case of the server kbeezie.com is on, I have a /64 block assigned to me by my datacenter. For those not familiar, a /64 block equates to about 18 quintillion addresses. In SolusVm you normally add a range of IPv6 addresses by randomly generating about a 100 addresses at a time. But you can also manually define an address like I did with kbeezie, choosing 2001:4858:aaaa:6::6 for this site. The :: represents an area that is filled with zeros, as such the long address would be 2001:4858:aaaa:0006:0000:0000:0006.

Now to add this address to a container we would simply use vzctl:

vzctl set <id> --ipadd <ipv6_addr> --save

For example if your containers’ CTID is 101, and you wanted to add 2001:4858:aaaa:6::1, you would perform the following:

vzctl set 101 --ipadd 2001:4858:aaaa:6::1 --save

Verifying IPv6 Connectivity

Now when you log into your container, you should be able to perform the following:

ping6 ipv6.google.com

If you get an error that there’s no connection to host, or you simply time out, then first double check that the IPv6 address shows in your ifconfig output. If it’s there then try the following command to add a default route from the container to the host node:

On CentOS/RHEL container:

ip route add 2000::/3 dev venet0

On Debian/Ubuntu container:

ip route add ::/0 dev venet0

You can verify the default route has been added with:

ip -6 route show

Once that has been added, then try to ping6 ipv6.google.com once a gain. If you see ping responses then you’re done with that. If not move onto the next step.

Adding route to container at physical node

Sometimes when you add an IP address to a container, the traffic simply doesn’t make it to the containers. I’ve had this annoyance a number of times especially when I add an IP address to a container in SolusVm.

Lets say for example you added 2001:4858:aaaa:6::1 to a container, but traffic simply is not making it thru. You may need to re-add the route between the container and the eht0 adapter. To do this use the following command:

ip -6 neigh add proxy 2001:4858:aaaa:6::1 dev eth0

Once thats done, log back into the container and try to ping6 ipv6.google.com again. At this point you should be getting a response.

Configuring Web Servers

As mentioned in the openvz wiki, link local and other features will not be available with a venet adapter. A prime example of this is listening on all available IPv6 interfaces such as [::]:80 will not work in most cases as the software will often complain of the address already being bound. Likewise any services that can listen on both IPv4 and IPv6 will need to be set to only handle IPv6 traffic over IPv6 (since its possible for IPv6 to handle both IPv6 and IPv4 traffic).

It helps to have Google’s nameservers ( 8.8.8.8 and 8.8.4.4 ) in your /etc/resolv.conf if you do not already have a IPv6-friendly nameserver configured.

The following are tweaks you can use to make IPv6 work in various webservers.

Nginx

Before you can use IPv6 with Nginx, the webserver needs to be compiled with the –with-ipv6 option. You may already have this option enabled, you can check by calling nginx -V. To listen on IPv6 you can add the following line to your server block:

listen [2001:4858:aaaa:6::1]:80 ipv6only=on;

Remember that each server block will need to have it’s own unique IPv6 address and that you cannot bind to all [::] interfaces. The option ipv6only=on needs to be enabled in order to play nicely with venet. If the physical node already has ipv4 binding turned off then ipv6only may not be necessary, but its a good idea to use anwyays with the venet adapter. To ensure that a server block is listening on both IPv6 and IPv4 you may need to have the following:

listen 0.0.0.0:80;
listen [2001:4858:aaaa:6::1]:80 ipv6only=on;

You can also choose to listen on a specific IPv4 address. The listen line for IPv4 is needed, since otherwise the server block will only be listening on IPv6.

More information on Nginx’s listen directive can be found in the Wiki.

Lighttpd

I’m not entirely familiar with lighttpd myself, but it does share a similar directive to nginx for forcing IPv6 listeners to only handle IPv6 traffic. IPv6 setup is outlined in Wiki overview. Which shows V6_ONLY is already enabled in most common cases where you have to statically define the IP address.

Again you cannot bind to [::] when using venet on OpenVz.

Apache/httpd

Apache does not have a v6-only type of directive. Instead you have to recompile apache with –disable-v4-mapped. Once Apache has been compiled you can use the [IPv6]:80 format, you’ll need to have a Listen directive set to your new IPv6 address, as well as any virtual hosts. Like with both nginx and lighttpd above, you cannot bind to all interfaces ([::]), only specifically assigned addresses.

Control panels, Firewalls, etc

Firewall software such as UFW (uncomplicated firewall by ubuntu), and such do not play nicely with IPv6 and OpenVz (over venet). So thats something to keep in mind when setting up firewall software that rely on kernel modules and such that are not often available in a virtualized environment (Xen is typically more suited for something like that).

Not all control panels are IPv6 ready. DirectAdmin for example does have the option to enable IPv6, and will let you manage IPv6 addresses in the IP configuration and DNS area. However it does not configure the services like apache, bind9, dovecot and so forth for you. For those services you’ll have to configure manually.

Conclusion

For the purpose of hosting websites and logging into my server via SSH, the above guidelines work just fine. I currently host kbeezie.com, kbeezie.net (static CDN), ionvz.com and a few other websites behind IPv6 with Nginx on such a configuration.

If you do not have IPv6 connectivity from your home or office machine, you can use a free tunnnel service such as HE.net’s IPv6 Tunnel Broker Service. which I’m using here at home. I currently use Hurricane Electric for my servers backbone (native IPv6 for my servers), Home IPv6 Tunneling, and Free DNS Service which seems to work much better than using my domain registrar’s DNS.

Apache and Nginx Together

January 30th, 2011

I’m not going to get into a lot of details about how to install and configure either http server from scratch. This article is primarily going to be food for thought for those who may want or need to configure nginx along side an existing apache (httpd) configuration. If you would rather ditch Apache all together, consider reading Martin Fjordvald’s Nginx Primer 2: From Apache to Nginx.

Side by Side

In some cases you may need to run both Apache (httpd) and Nginx on port 80. Such a situation can be a server running Cpanel/Whm and as such doesn’t support nginx, so you wouldn’t want to mess with the apache configuration at all.

To do this you have to make sure Apache and Nginx are bound to their own IP adddress, In the event of WHM/Cpanel based webserver, you can Release an IP to be used for Nginx in WHM. At this time I am not aware of a method of reserving an IP, and automatically forcing Apache to listen on a specific set of IPs in a control panel such as DirectAdmin or Plesk. But the link above will show you how with WHM/Cpanel.

Normally Apache will be listening on all interfaces, and such you may see a line like this in your httpd.conf file:

Port 80
#or
Listen 80

Instead you’ll need to tell apache to listen on a specific IP (you can have multiple Listen lines for multiple IPs)

Listen 192.170.2.1:80

When you change Apache to listen on specific interfaces some VirtualHost may fail to load if they are listening on specific IPs themselves. The address option in <VirtualHost addr[:port] [addr[:port]] …> only applies when the main server is listening on all interfaces (or an alias of such). If you do not have an IP Listed in your <VirtualHost …> tag, then you do not need to worry bout this. Otherwise make sure to remove the addr:port from the VirtualHost tag.

Make sure to update the NameVirtualHost directive in Apache for name-based virtual hosts.

Once you have apache configured to listen on a specific set of IPs you can do the same with nginx.

server {
    listen 192.170.2.2:80;
    ...
}

Now that both servers are bound to specific IPs, both can then be started up on port 80. From there you would simply point the IP of the domain to the server you wish to use. In the case of WHM/Cpanel you can either manually configure the DNS entry for the domain going to nginx in WHM, or you can use your own DNS such as with your registrar to point the domain to the specific IP. Using the latter may break your mail/ftp etc configurations if the DNS entries are not duplicated correctly.

Though normally if you are setting up Nginx side-by-side with apache, you’ll have your domain configurations separate from the rest of the system. The above paragraph only applies if the domain you are using has already been added by cpanel/whm and you wish to have it served by nginx exclusively. [Scroll down for PHP considerations]

Apache behind Nginx

If you are using a control panel based hosting such as cpanel/whm, this method is not advised. Most of the servers configuration is handled automatically in those cases, and making manual changes will likely lead to problems (plus you won’t get support from the control panel makers or your hosting provider in most cases).

Using Nginx as the primary frontend webserver can increase performance regardless if you choose to keep Apache running on the system. One of Nginx’s greatest advantage is how well it serves static content. It does so much more efficiently than Apache, and with very little cost to memory or processing. So placing Nginx in front will remove that burdern off Apache, leaving it to concentrate on dynamic request or special scenarios.

This method is also popular for those who don’t want to use PHP via fastcgi, or install a separate php-fpm process.

First thing that needs to be done is to change the interface apache listens on:

Listen 127.0.0.1:8080

Above we bind Apache to the localhost on an alternate port, since only Nginx on the same machine will be communicating with Apache.

Note: Since Nginx needs to access the same files that Apache serves, you need to make sure that Nginx is setup to run as the same user as apache, or to make sure that the Nginx’s selected user:group has permission to read the web documents. Often times the webroot is owned by nobody or www-data and Nginx likewise can be setup to run as those users

Then in Nginx listening on port 80 (either on all interfaces such as 0.0.0.0 or to specific IPs, your choice) we would need to configure Nginx to serve the same content. Take for example this virtual host in an apache configuration:

<VirtualHost>
      DocumentRoot "/usr/local/www/mydomain.com"
      ServerName mydomain.com
      ServerAlias www.mydomain.com
      CustomLog /var/log/httpd/mydomain_access.log common
      ErrorLog /var/log/httpd/mydomain_error.log
      ...
</VirtualHost>

In Nginx the equivalent server configuration would be:

server {
      root /usr/local/www/mydomain.com;
      server_name mydomain.com www.mydomain.com;
 
      # by default logs are stored in nginx's log folder
      # it can be changed to a full path such as /var/log/...
      access_log logs/mydomain_access.log;
      error_log logs/mydomain_error.log;
      ...
}

The above would serve all static content from the same location, however PHP will simply come back as PHP source. For this we need to send any PHP requests back to Apache.

server {
      root /usr/local/www/mydomain.com;
      server_name mydomain.com www.mydomain.com;
 
      access_log logs/mydomain_access.log;
      error_log logs/mydomain_error.log;
 
      location / { 
            # this is the location block for the document root of this vhost
      } 
 
      location ~ \.php {
            # this block will catch any requests with a .php extension
            # normally in this block data would be passed to a FastCGI process
 
            # these two lines tell Apache the actual IP of the client being forwarded
            # Apache requires mod_proxy (http://bit.ly/mod_proxy) for these to work
            # Most Apache 2.0+ servers have mod_proxy configured already
 
            proxy_set_header X-Real-IP  $remote_addr;
            proxy_set_header X-Forwarded-For $remote_addr;
 
            # this next line adds the Host header so that apache knows which vHost to serve
            # the $host variable is automatically set to the hostname Nginx is responding to
 
            proxy_set_header Host $host;
 
            # And now we pass back to apache
            # if you're using a side-by-side configuration the IP can be changed to
            # apache's bound IP at port 80 such as http://192.170.2.1:80
 
            proxy_pass http://127.0.0.1:8080;
      }
 
       # if you don't like seeing all the errors for missing favicon.ico in root
       location = /favicon.ico { access_log off; log_not_found off; }
 
       # if you don't like seeing errors for a missing robots.txt in root
       location = /robots.txt { access_log off; log_not_found off; }
 
       # this will prevent files like .htaccess .htpassword .secret etc from being served
       # You can remove the log directives if you wish to
       # log any attempts at a client trying to access a hidden file
       location ~ /\. { deny all; access_log off; log_not_found off; }
}

In the case of rewriting (mod_rewrite) with apache behind nginx you can use an incredibly simple directive called try_files. Here’s the same code as above, but when changes made so that any files or folder not found by nginx gets passed to apache for processing (useful for situations like WordPress, or .htaccess based rewrites when the file or folder is not caught by Nginx)

server {
      root /usr/local/www/mydomain.com;
      server_name mydomain.com www.mydomain.com;
 
      access_log logs/mydomain_access.log;
      error_log logs/mydomain_error.log;
 
      location / { 
            # try_files attempts to serve a file or folder, until it reaches the fallback at the end
            try_files $uri $uri/ @backend;
      } 
 
      location @backend {
            # essentially the same as passing php requests back to apache
 
            proxy_set_header X-Real-IP  $remote_addr;
            proxy_set_header X-Forwarded-For $remote_addr;
            proxy_set_header Host $host;
            proxy_pass http://127.0.0.1:8080;
      }
      # ... The rest is the same as the configuration above ... 
}

With the above configuration, any file or folder that exists will be served by Nginx (the php block will catch files ending in .php), otherwise it will be passed back to apache on the backend.

In some cases you may wish for everything to be passed to Apache unless otherwise specified, in which case you would have a configuration such as this:

server {
      root /usr/local/www/mydomain.com;
      server_name mydomain.com www.mydomain.com;
 
      access_log logs/mydomain_access.log;
      error_log logs/mydomain_error.log;
 
      location / {
            proxy_set_header X-Real-IP  $remote_addr;
            proxy_set_header X-Forwarded-For $remote_addr;
 
            proxy_set_header Host $host;
 
            proxy_pass http://127.0.0.1:8080;
      }
 
      # Example 1 - Specify a folder and it's content
      # To serve everything under /css from Nginx-only
      location /css { }
 
      # Example 2 - Specify a RegEx pattern such as file extensions
      # Serve certain files directly from Nginx
      location ~* ^.+\.(jpg|jpeg|gif|png|css|zip|pdf|txt|js|flv|swf|html|htm)$
      {
            # this will basically match any file of the above extensions
            # course if the file does not exist you'll see an Nginx error page as opposed
            # to apache's own error page. 
      }
}

Personally I prefer to leave Nginx to handle everything, and specify exactly what needs to be passed back to Apache. But the last configuration above would more accurately allow .htaccess to function during almost-every request since almost everything gets passed back to apache. The css folder or any files found by the other location block would be served directly by nginx regardless if there’s an .htaccess file there, since only Apache processes .ht* files.

PHP Considerations

If you are not familiar with Nginx, then it should be noted that Nginx does not have a PHP module like apache’s mod_php, instead you either need to build PHP with FPM (ie: php-fpm/fastcgi), or you need to pass the request to something that can handle PHP.

In the case of using Nginx with Apache you essentially have two choices:

1) Compile and Install PHP-FPM, thus running a seperate PHP process to be used by Nginx. This option is usually good if you want to keep the two webserver configurations separated from each other, that way any changes to one won’t affect the other. But of course it adds additional memory usage to the system.

2) Utilize Apache’s mod_php. This option is ideal when you will be passing data to apache as a backend. Even in a side-by-side scenario, you can utilize mod_php by proxying any php request to Apache’s IP. But in the side-by-side scenario you have to make sure that Apache is also configured to serve the same virtualhost that Nginx is requesting.

Speed-wise both methods are about the same when PHP is configured with the same set of options on both. Which option you choose depends solely on your needs. Another article that may be of interest in relations to this one would be Nginx as a Proxy to your Blog, which can be just as easily utilized on a single server (especially if you get multiple IP ranges like I do).

Apache to Nginx Migration Tips

April 11th, 2010

For a simple beginning example configuration to start with please view my example configuration

Nginx currently holds shy of 6.5% of the known webserver market, which is just roughly shy of 13 million servers. This little lightweight webserver created by a sole Russian developer has been gaining a great deal of popularity over the last few years and is used by sites such as WordPress, Texts from Last Night and Hulu.

This guide will show you common examples in Nginx to make migration from Apache a bit easier.

HTTP Basic Authentication or htpasswd support

If you have been playing with nginx for a the first time you may have noticed that there is no .htaccess support. However Nginx can still utilize HTTP Basic Authenication with existing htpasswd files using the HTTP Auth Basic Module. The module is compiled by default.

If you still have Apache installed (but perhaps turned off or on a different port) you can still use htpasswd to generate a password file. But in the event that you don’t have or don’t want to install Apache you can use Perl or Ruby to generate at least the encrypted passwords. The .htpasswd files should be in the following format.

username:encrypted-password:comment

To generate a password with Ruby in irb:

"your-password".crypt("salt-hash")

Likewise to generate with perl from the terminal:

perl -le 'print crypt("your-password", "salt-hash")'

The two commands above will utilize a 56-bit DES encryption which can be used in the htpasswd file.

Once you have your password file saved, store it outside of the web-accessible location like you would have when using Apache (likewise you can ensure that web access to hidden files are denied, more on that later).

Lets say you have a subfolder on your server called ‘admin’ and you wish to protect it. Here is an example configuration:

location  /admin  {
  auth_basic            "Admin Access";
  auth_basic_user_file  conf/domain.com/admin/htpasswd;
}

If the above did not work, check your error logs as sometimes it will notify you if the path to the password file could not be found.

Ignoring robots.txt, favicon.ico and hidden files

Sometimes you may wish to omit commonly accessed files from being recorded in the access log as well as block access to hidden files (filenames beginning with a period such as .htpasswd and .htaccess). Even though nginx doesn’t support htaccess, you may still wish to secure files left behind from the migration. You can easily do this by adding a couple location blocks to your domain’s server block.

location = /favicon.ico { access_log off; log_not_found off; }	
location = /robots.txt { access_log off; log_not_found off; }
location ~ /\. { deny  all; access_log off; log_not_found off; }

The above will not record access to the favicon.ico or robots.txt in the root of your site, as well as ignore the file-not-found error if you do not have either file present. Also the last line will deny access to any file beginning with a period anywhere within the root path. You can actually save the above into a file such as /conf/drop.conf, and include it at the bottom of each of your server blocks like so.

include drop.conf;

Extremely Easy WordPress Migration

Most people now days take advantage of WordPress’ permalink feature which normally requires mod_rewrite enabled in the .htaccess file. However as of Nginx 0.7.26 and above this can be taken care of very easily with a single directive.

location / { 
   try_files $uri $uri/ index.php;
}

The above will attempt to try the requested URI first as a file, then as a folder, and if both of those come back negative it will fall back to index.php. WordPress can simply parse the request URI sent by the webserver so there is no need to manually add arguments onto index.php. If you instead need to do something further you can change the fallback from index.php to @fallback like so:

location / {
   try_files $uri $uri/ @wordpress;
}
 
location @wordpress {
     # Additional rewrite rules or such you need here.
}

For Nginx versions older than 0.7.26 you can use the older method shown below (however its strongly advised to have 0.7.26 or newer).

location / { 
        if (-f $request_filename) {
            expires 30d;
            break;
        }
 
        #Ultimately 'if' should only be used in the context of rewrite/return : http://wiki.nginx.org/IfIsEvil
         if (!-e $request_filename) {
            rewrite ^(.+)$ /index.php?q=$1 last;
        }
}

The two if blocks replace the common RewriteCond used by wordpress to test if the requested filename is an existing file or folder.

On the next page: Utilizing WP Super Cache, rewrite examples and more.

Handling wildcard subdomains with PHP

November 5th, 2009

Subdomains can be a very handy way to make your urls more friendly looking. They can also be incredibly useful for membership driven websites to allow members to have their own custom subdomain. But how do make manage dynamic subdomains with PHP?

There is two parts to making dynamic subdomains work in this case. The first is updating your DNS and webserver to expect wildcard requests. On the DNS side its a simple matter of adding * as an ‘A’ record, or cname pointing to the primary domain or IP. In most cases if you are using shared hosting, you’ll need to contact your hosting provider to enable wildcard DNS on your account. Otherwise if you are on a dedicated server or virtual private server, this can be most likely handled by yourself, such as editing the DNS zone in WHM.

Once you have enabled wildcard DNS, you’ll need to configure your webserver to expect a *.domain.com request. Otherwise it will ignore all requests other than the base domain and pre-defined subdomains.

In Nginx simply modify the server_name directive:

server_name yourdomain.com *.yourdomain.com;

In Apache you would add a ServerAlias directive to the appropriate virtual host entry.

ServerAlias *.yourdomain.com

On most Cpanel driven servers the http.conf file for Apache can be found in /usr/local/apache/conf.

Once you have configured both the name server and web server for wildcard requests you can now use php to not only detect the provide subdomain, but also generate them. The code below will obtain the subdomain being visited.

$sub = str_replace(".yourdomain.com", "", $_SERVER["HTTP_HOST"]);

The above will simply take the hostname stored in the HTTP_HOST enviroment variable, and attempt to replace the base domain with nothing (if no subdomain was provided all that remains is the base domain).

Once the subdomain is determined, you can filter out requests either with an if/then block or in a switch/case statement. In the event of using memeber names as subdomains you can use a switch statement to make sure that the subdomain doesn’t match the homepage (www / yourdomain.com) or a reserved page name such as register.yourdomain.com and assume anything else must be a member name. With that information you can pull the members information from a file or database.

Using PHP to handle the subdomain is certainly easier for most novice as opposed to modifying an htaccess to do rewrite rules passing the subdomain off as a query (which takes more resources than str_replace) or having to modify the webservers configuration file, especially if you do not have direct access to the webserver’s configuration file or do not have override permission.

If your current hosting provider does not support wildcard subdomain, and you are running a memeber based site, consider upgrading to a Virtual Private Server or use a shared hosting provider such as HostGator that allows you to turn on such capability via a support ticket.