Setting up Varnish caching server for pd-admin with Apache and php-fcgi

Deploying Varnish caching server can significantly speed up web-based applications, as well as simple websites. Varnish supports different back-end systems, ranging from the popular Apache web server to the more efficient nginx. Especially when it comes to handling high traffic sites Varnish can bring a considerable uplift to the responsiveness by e.g. caching entire pages (i.e. full-page cache).

In order to improve the responsiveness of a couple of high traffic websites running on pd-admin (an advanced collection of tools to administrate web- and mail hosting on Linux-based servers) Varnish 4 was implemented. The following steps describe the process of setting up Varnish caching server for pd-admin with Apache and php-fcgi on a Debian 7 server.

Setup Varnish on Debian 7 64bit

Let’s have a quick look at the process of installing varnish on a Debian 7 64 bit machine. Basically, you need to do 5 things:

1. Install https support for apt-get
2. Add GPG key for apt-get
3. Add the repository to apt-get packages source list
4. Refresh package source list
5. Install Varnish

Thus, in order to setup Varnish on Debian 7 64 bit execute the following commands:

1. apt-get install apt-transport-https
2. curl https://repo.varnish-cache.org/GPG-key.txt | apt-key add –
3. echo “deb https://repo.varnish-cache.org/debian/ wheezy varnish-4.0” >> /etc/apt/sources.list.d/varnish-cache.list
4. apt-get update
5. apt-get install varnish

That’s it. Varnish should be installed on ready to be configured.

Configure Varnish through VCL – default.vcl

Now that Varnish is installed it’s time to configure it. This can be done by editing default.vcl – the default configuration file automatically created during the installation:

#
# This is an example VCL file for Varnish.
#
# It does not do anything by default, delegating control to the
# builtin VCL. The builtin VCL is called when there is no explicit
# return statement.
#
# See the VCL chapters in the Users Guide at https://www.varnish-cache.org/docs/
# and http://varnish-cache.org/trac/wiki/VCLExamples for more examples.

vcl 4.0;

backend default {
 .host = "127.0.0.1";
 .port = "80";
}

sub vcl_recv {
 # Happens before we check if we have this in cache already.
 #
 # Typically you clean up the request here, removing cookies you don't need,
 # rewriting the request, etc.
}

sub vcl_backend_response {
 # Happens after we have read the response headers from the backend.
 #
 # Here you clean the response headers, removing silly Set-Cookie headers
 # and other mistakes your backend does.
}

sub vcl_deliver {
 # Happens when we have all the pieces we need, and are about to send the
 # response to the client.
 #
 # You can do accounting or modifying the final object here.
}

As starting point have a look at the very handy Varnish 4.0 configuration template which works out of the box for

• WordPress
• Drupal (works decently for Drupal 7, depends on your modules obviously)
• Joomla (WIP)
• Fork CMS
• OpenPhoto

and additional configuration setups like

• Server-side URL rewriting
• Clean error pages for debugging
• Virtual Host implementations
• Various header normalizations
• Cookie manipulations
• 301/302 redirects from within Varnish

Have a look at the Varnish configuration documentation to adjust it to your specific needs. In general, only few adaptions should be required using this template.

Set correct backend hostname to solve 403 forbidden error code

In case you get a 403 forbidden error make sure to set the correct backend hostname or IP in default.vcl:

backend server1 { # Define one backend
  .host = "USE_DOMAIN_SET_IN_HTTPD_CONF"; # IP or Hostname of backend
  ...
}

Have a look at pd-admin’s httpd.conf for the currently set hostname to be used by Varnish, or open up the web administration console and check the server name option. Also make sure to have set the X-Forwarded-For header (especially for Varnish 3):

req.http.x-forwarded-for = client.ip

That should solve the 403 forbidden error for cached domains.

Exclude Domains from Varnish Caching

Varnish by default caches all requests that are not excluded specifically. Thus, in case you want to exclude domains and simple URLs from being cached by Varnish you can specify them in vcl_recv() function like so:

if (req.http.host == "www.domain.com" && req.url == "/") {
return (pass);
}

This will redirect specific domains and/or URLs to the pass() function thus by-passing the caching mechanism.

Change Varnish port

Finally, you need to set Varnish to listen on the default http port (i.e. 80) and change Apache’s listener port to something different, e.g. 81 and set it as the Varnish backend port in default.vcl. Make sure to restart Apache and Varnish. That’s it!