Posted on Leave a comment

Setting up Varnish caching server for pd-admin with Apache and php-fcgi

Deploying Varnish caching server can significantly speed up web-based applications, as well as simple websites. Varnish supports different back-end systems, ranging from the popular Apache web server to the more efficient nginx. Especially when it comes to handling high traffic sites Varnish can bring a considerable uplift to the responsiveness by e.g. caching entire pages (i.e. full-page cache).

In order to improve the responsiveness of a couple of high traffic websites running on pd-admin (an advanced collection of tools to administrate web- and mail hosting on Linux-based servers) Varnish 4 was implemented. The following steps describe the process of setting up Varnish caching server for pd-admin with Apache and php-fcgi on a Debian 7 server.

Setup Varnish on Debian 7 64bit

Let’s have a quick look at the process of installing varnish on a Debian 7 64 bit machine. Basically, you need to do 5 things:

  1. Install https support for apt-get
  2. Add GPG key for apt-get
  3. Add the repository to apt-get packages source list
  4. Refresh package source list
  5. Install Varnish

Thus, in order to setup Varnish on Debian 7 64 bit execute the following commands:

  1. apt-get install apt-transport-https
  2. curl https://repo.varnish-cache.org/GPG-key.txt | apt-key add –
  3. echo “deb https://repo.varnish-cache.org/debian/ wheezy varnish-4.0” >> /etc/apt/sources.list.d/varnish-cache.list
  4. apt-get update
  5. apt-get install varnish

That’s it. Varnish should be installed on ready to be configured.

Configure Varnish through VCL – default.vcl

Now that Varnish is installed it’s time to configure it. This can be done by editing default.vcl – the default configuration file automatically created during the installation:

#
# This is an example VCL file for Varnish.
#
# It does not do anything by default, delegating control to the
# builtin VCL. The builtin VCL is called when there is no explicit
# return statement.
#
# See the VCL chapters in the Users Guide at https://www.varnish-cache.org/docs/
# and http://varnish-cache.org/trac/wiki/VCLExamples for more examples.

vcl 4.0;

backend default {
 .host = "127.0.0.1";
 .port = "80";
}

sub vcl_recv {
 # Happens before we check if we have this in cache already.
 #
 # Typically you clean up the request here, removing cookies you don't need,
 # rewriting the request, etc.
}

sub vcl_backend_response {
 # Happens after we have read the response headers from the backend.
 #
 # Here you clean the response headers, removing silly Set-Cookie headers
 # and other mistakes your backend does.
}

sub vcl_deliver {
 # Happens when we have all the pieces we need, and are about to send the
 # response to the client.
 #
 # You can do accounting or modifying the final object here.
}

As starting point have a look at the very handy Varnish 4.0 configuration template which works out of the box for

  • WordPress
  • Drupal (works decently for Drupal 7, depends on your modules obviously)
  • Joomla (WIP)
  • Fork CMS
  • OpenPhoto

and additional configuration setups like

  • Server-side URL rewriting
  • Clean error pages for debugging
  • Virtual Host implementations
  • Various header normalizations
  • Cookie manipulations
  • 301/302 redirects from within Varnish

Have a look at the Varnish configuration documentation to adjust it to your specific needs. In general, only few adaptions should be required using this template.

Set correct backend hostname to solve 403 forbidden error code

In case you get a 403 forbidden error make sure to set the correct backend hostname or IP in default.vcl:

backend server1 { # Define one backend
  .host = "USE_DOMAIN_SET_IN_HTTPD_CONF"; # IP or Hostname of backend
  ...
}

Have a look at pd-admin’s httpd.conf for the currently set hostname to be used by Varnish, or open up the web administration console and check the server name option. Also make sure to have set the X-Forwarded-For header (especially for Varnish 3):


req.http.x-forwarded-for = client.ip

That should solve the 403 forbidden error for cached domains.

Exclude Domains from Varnish Caching

Varnish by default caches all requests that are not excluded specifically. Thus, in case you want to exclude domains and simple URLs from being cached by Varnish you can specify them in vcl_recv() function like so:


if (req.http.host == "www.domain.com" && req.url == "/") {
return (pass);
}

This will redirect specific domains and/or URLs to the pass() function thus by-passing the caching mechanism.

Change Varnish port

Finally, you need to set Varnish to listen on the default http port (i.e. 80) and change Apache’s listener port to something different, e.g. 81 and set it as the Varnish backend port in default.vcl. Make sure to restart Apache and Varnish. That’s it!

Posted on Leave a comment

How to solve “550 sorry, user unknown” problem when using pd-admin

Source Code Icon

Recently, a client reported problems when trying to send mail to certain domains hosted on their own web server. The web server in question was running pd-admin as administration tool set. Mail hosting was done on a separate dedicated mail server. The error message reported by the client was the infamous

550 sorry, user unknown

Checking SMTP through telnet

Having a quick look at the pd-admin (web server) and qmail configuration (mail server) did not reveal any errors in the setup. Thus, a quick telnet check for the email account in question was done to make sure the SMTP service was operating normally:

COMMAND:
telnet mx1.company.com 25
RESPONSE:
Connected to mx1.company.com.
Escape character is ‘^]’.
220 mx1.company.com ESMTP

COMMAND:
helo hi
RESPONSE:
250 mx1.company.com

COMMAND:
mail from: <your-email@gmail.com>
RESPONSE:
250 2.1.0 Ok

COMMAND:
rcpt to: <user.in.question@company.com>
RESPONSE:
550 5.1.1 <user.in.question@company.com>: Recipient address rejected: User unknown in virtual alias table

COMMAND:
quit
RESPONSE:
221 2.0.0 Bye

This check was done on the troubled web server itself. It seemed like that the MX record was pointing to the web server itself and not the mail server.

The solution

When creating customer accounts and setting up domains through pd-admin it automatically creates MX records for these domains too (although the mail server option was not selected during the domain setup on the web server). In case you want to get rid of these MX records

  • navigate to Customers

pd-admin-customers

  • select Overview and select the Domain option for the customer in question

pd-admin-customers-overview

  • then select the MX option

pd-admin-customers-mx

  • and finally deactivate the MX record

pd-admin-customers-mx-off   That should solve it! Yes, a pretty simple solution 😉