Blog

Magento Error ESI processing not enabled

In a recent Magento setup with a Varnish caching server an undefined PHP variable in a template resulted in the error message “ESI processing not enabled“.

Due to the undefined PHP variable the underlying Varnish caching server’s Edge-side includes (ESI) capabilities, allowing content assembly by HTTP surrogates through in-markup XML-based language failed. After a quick code review of a just installed Magento extension there existed a newly introduced undefined PHP variable that caused Varnish to fail.

Thus, when using Varnish double check for undefined variables in Magento code 😉

Remove Passphrase from SSL Keys

When it comes to generating SSL keys passphrases play an immanent role. They need to be specified when creating SSL keys and are checked each time the key is being used to ensure authorized access. For instance, when starting your Apache web server with a SSL certificate you will need to enter the original passphrase to verify authorized access. The following simple step shows you how to remove passphrase from SSL keys.

If you don’t already have a SSL key create a 2048 bit RSA key with triple DES block ciphering first and specify your passphrase as usual:

openssl genrsa -des3 -out your-server.key 2048

Of course you can choose any other modulus bits count and ciphering mode to generate your SSL key. Then, make a backup of the original certificate with the passphrase still set just in case:

cp your-server.key your-server.key.WITH_PASS

Remove Passphrase

And finally remove passphrase from your SSL key:

openssl rsa -in your-server.key.WITH_PASS -out your-server.key.WITHOUT_PASS

Now you can use this key without requiring the enter the passphrase on every single use, e.g. when Apache web server starts, etc. That’s it.

Display localized dates in Magento

Since Magento uses the Zend Framework you are able to use Zend_Date to easily display localized dates in Magento.

$date = new Zend_Date(Mage::getModel('core/date')->timestamp(time(), false, Mage::app()->getLocale()->getLocaleCode()));
echo $date->toString("EEEE, d. MMMM YYYY");

So, first create a Zend_Date object based on the locale set in the administration backend:

Mage::app()->getLocale()->getLocaleCode()

Then simply output your date as localized string using Zend_Date->toString:

echo $date->toString("EEEE, d. MMMM YYYY");

Us usual, find formating rules in the Zend_Date constants documentation.

Enabling Cross-Origin Resource Sharing CORS for Apache

When trying to share resources across different domains (host-a.com vs. host-b.com) you will come across the concept of Cross-Origin Resource Sharing (CORS). In order to be able to share resources across different domains you will most likely need to enable to CORS manually on your server. For instance, when a script on host-a.com tries to access resources on host-b.com CORS needs to be enabled on host-b.com. This post shows you the steps needed for enabling Cross-Origin Resource Sharing CORS for Apache using a .htaccess file.

Note: Looking for a way to enable CORS for PHP? Have a look at Enabling Cross-Origin Resource Sharing CORS for PHP.

Prepare your .htaccess

In order to to enable CORS on Apache you need mod_headers and mod_setenvif. The example below show you how to prepare Apache for the two types of CORS requests:

  1. Pre-Flight Requests
  2. Simple Requests

Pre-Flight Requests are generated on the client side when not using GET, POST and HEAD requests on external domain resources (or setting custom headers). In this case we need to check Access-Control-Allow-Headers and Access-Control-Allow-Methods in order to allow or disallow CORS requests to our server. Second, simple requests handle GET, POST and HEAD requests across different domains. In this case we need to check the Origin header and set Access-Control-Allow-Origin correspondingly to allow such CORS requests. Below you find examples to allow CORS for method-a Pre-Flight Requests and simple requests from host-a.local or host-b.local, optionally for accessing different remote scripts too (e.g. https://host-b.local/method-a.php).



   ##########################################################################
   # 1.) ENABLE CORS PRE-FLIGHT REQUESTS
   # e.g. PUT, DELETE, OPTIONS, ...
   # we need to set Access-Control-Allow-Headers and 
   # Access-Control-Allow-Methods for allowed domain(s)
   ##########################################################################
   
   # first check for pre-flight headers and set as environment variables
   # e.g. header method-a is set here
   SetEnvIf ^Access-Control-Request-Method$ "method-a" METHOD_A
   SetEnvIf ^Access-Control-Request-Headers$ "^Content-Type$" HEADER_A

   # set corresponding response pre-flight headers for allowed domain(s)
   Header set Access-Control-Request-Methods "method-a" env=METHOD_A
   Header set Access-Control-Request-Headers "content-type" env=HEADER_A
   
   # TODO: add allowed additional pre-flight requests here...
   
   #########################################################################
   # 2.) ENABLE CORS *SIMPLE REQUESTS* (vs. Pre-Flight Requests from above)
   # e.g. GET, POST and HEAD requests
   # we need to set Access-Control-Allow-Origin header for allowed domain(s)
   # also note that POST requests need to match one of the following 
   # Content-Type: 
   # 1) application/x-www-form-urlencoded
   # 2) multipart/form-data
   # 3) text/plain
   #########################################################################
   
   # write line for each domain you would like to enable CORS requests for
   # e.g. origin = http://host-a.local or http://host-b.local 
   SetEnvIfNoCase Origin "((http(s?))?://(www\.)?(host\-a|host\-b)\.local)(:\d+)?$" AccessControlAllowOrigin=$0
   Header set Access-Control-Allow-Origin %{AccessControlAllowOrigin}e env=AccessControlAllowOrigin
   
   # e.g. origin = https://host-b.local/method-a.php
   SetEnvIfNoCase Origin "https://host-b.local/method-a.php" AccessControlAllowOrigin=$0
   Header set Access-Control-Allow-Origin %{AccessControlAllowOrigin}e env=AccessControlAllowOrigin
   
   # generic regexp match for more flexibel use cases
   #SetEnvIfNoCase Origin "((http(s?))?://(www\.)?(host\-a|host\-b)\.local)(:\d+)?$" AccessControlAllowOrigin=$0
   #Header set Access-Control-Allow-Origin %{AccessControlAllowOrigin}e env=AccessControlAllowOrigin
 
   # TODO: add additional allowed simple requests here...
 

Test File

In case you want to test this CORS setup here’s a simple test file:

<script src="//code.jquery.com/jquery-1.11.0.min.js"></script><script>// <![CDATA[
function a() {
var jqxhr = $.get( "http://host-b.local", function() {
})
.done(function() {
alert( "second success" );
})
.fail(function() {
alert( "error" );
})
}

function b() {
var jqxhr = $.get( "http://host-b.local/method-b", function() {
})
.done(function() {
alert( "second success" );
})
.fail(function() {
alert( "error" );
})
}
// ]]></script><a>Host B (GET)</a> <a>Host B / Method A (GET)</a>

Be sure to configure your Apache VHost settings accordingly.

Improvements

Since we don’t want to manually edit .htaccess files a simple configuration script would be nice to set domain configurations automatically for CORS enabled domains/methods.

Simple Random Posts Shortcode WordPress Plugin

A simple but effective WordPress plugin to render random blog posts in pages and/or posts. This is the first public release and features the following shortcode options to customize the rendering process:

  1. header: whether to output header text (default=)
  2. show_featured_image: whether to output featured post image (default=TRUE)
  3. featured_image_height: height of featured image in px (default=100)
  4. preview_text_chars: amount of chars to be displayed for preview text (default=200)
  5. container_css_class: CSS class of outer container (default=articles-preview-container)
  6. container_article_css_class: CSS class of article containe (default=article-preview)
  7. append_css_clear: whether to append a CSS clear container (default=TRUE)
  8. container_left_width: px or % (default=65%)
  9. container_right_width: px or % (default=30%)

Usage

In order to use the plugin just insert the shortcode

[simple_random_post]

and use the options provided above to customize the plugin. Feel free to download it from the official WordPress plugin repository.

Demo

Below you find a demo using the following shortcode:

simple_random_posts header="<b>Custom header</b>

[simple_random_posts header=”Custom header“]

Installing Android KitKat on HTC Desire

As shown in Installing Android Jelly Beans on HTC Desire Bravo you are able to install Android Jelly Beans on your HTC Desire by following a few simple steps. Thanks to Chromium, you are also able to install Android KitKat on your HTC Desire using a ROM called Beanstalk. The currently latest version 6 is based on Android 4.4.4 and offers great improvements (speed, memory handling, battery lifetime, etc.) over JellyBeans. In order to install this ROM you can follow all steps explained in the post Installing Android Jelly Beans on HTC Desire Bravo.

The steps needed are:

  1. root your device
  2. prepare your SD card
  3. deploy the ROM and Google Apps to your SD card
  4. install KitKat ROM
  5. optionally set up Mount2sd for better memory handling

Be sure to check Chromium’s XDA developers Beanstalk release page. Enjoy!

Simple modal HTML dialog overlay

In case you don’t want to deploy full fledged JavaScript libraries, such as jQuery or Dojo, to implement a simple modal HTML dialog overlay here is a quick and easy way to do so based on plain HTML, CSS and JavaScript. It works on all modern browsers.

First, let’s have a look at the CSS code:

#DialogOverlay {
    position: fixed;
    top: 0;
    left: 0;
    width: 100%;
    height: 100%;
    z-index: 99999;
    background: rgba(0, 0, 0, 0.5);
    display: none;
}
#DialogOverlayContent {
    position: absolute;
    top: 40%;
    left: 15%;
    color: #000;
    text-align: center;
    width: 70%;
    height: 200px;
    z-index: 999999;
    background: rgba(255, 255, 255, 1);
    border: 2px solid;
    border-radius: 25px;
}
#DialogOverlayContent > p {
    color: #000;
    margin-top: 50px;
}

And the corresponding JavaScript code:

function showFormDialog() {
    // using plain JavaScript
    document.getElementById('DialogOverlay').style.display = 'block';

    // using jQuery
    // $('#DialogOverlay').css('display', 'block');
}
function closeDialog() {
    // using plain JavaScript
    document.getElementById("DialogOverlay").style.display = 'none';

    // using jQuery
    // $("#DialogOverlay").css('display', 'none');
}

And finally the HTML code:

<div id="DialogOverlay">
  <div id="DialogOverlayContent">
    <p><img src="images/loading.gif" alt="" width="30" height="30" style="margin-right: 20px;"/>
    Please stand by while we process your request...<br/><br/>
    Please do not hit refresh, you will be redirected shortly!<br/><br/>
    <a href="#" onclick="closeDialog();">Close</a></p>
  </div>
</div>

A complete example can be found here:

<html>
<head>
<title>Simple modal HTML dialog overlay</title>
<script>
function showFormDialog() {
    // using plain JavaScript
    document.getElementById('DialogOverlay').style.display = 'block';

    // using jQuery
    // $('#DialogOverlay').css('display', 'block');
}
function closeDialog() {
    // using plain JavaScript
    document.getElementById("DialogOverlay").style.display = 'none';

    // using jQuery
    // $("#DialogOverlay").css('display', 'none');
}
</script>
<style>
#DialogOverlay {
    position: fixed;
    top: 0;
    left: 0;
    width: 100%;
    height: 100%;
    z-index: 99999;
    background: rgba(0, 0, 0, 0.5);
    display: none;
}
#DialogOverlayContent {
    position: absolute;
    top: 40%;
    left: 15%;
    color: #000;
    text-align: center;
    width: 70%;
    height: 200px;
    z-index: 999999;
    background: rgba(255, 255, 255, 1);
    border: 2px solid;
    border-radius: 25px;
}
#DialogOverlayContent > p {
    color: #000;
    margin-top: 50px;
}
</style>
</head>
<body>

<div id="DialogOverlay">
  <div id="DialogOverlayContent">
    <p><img src="images/loading.gif" alt="" width="30" height="30" style="margin-right: 20px;"/>
    Please stand by while we process your request...<br/><br/>
    Please do not hit refresh, you will be redirected shortly!<br/><br/>
    <a href="#" onclick="closeDialog();">Close</a></p>
  </div>
</div>

<a href="#" onclick="showFormDialog();">Open dialog</a>
</body>
</html>

That’s it, pretty simple but effective 🙂

Fixing ownCloud 6.0.3 upgrade problem on Asustor Data Master (ADM)

Unfortunately, trying to upgrade ownCloud on an Asustor AS-602T to version 6.0.3 running Asustor Data Master (ADM) 2.1.1.R3B2 resulted in a corrupted installation. Even removing and re-installing ownCloud via the App Central didn’t solve this issue.

So, in order to solve this issue and remove your corrupted ownCloud installation from your system first remove it via the App Central and then SSH to your NAS and remove the following folder:

NOTE: The following command will remove your installation and settings!

root@NAS:/volume1/.@plugins/AppCentral # rm -rf owncloud/

This fixed the ownCloud upgrade problem and I was able to install ownCloud 6.0.3.

Setup Varnish Cache Server for Magento 1.9 CE on Mac OSX

When dealing with high traffic Magento installations you definitely want to implement caching mechanisms to ensure proper response times for your visitors. Apart from internal caching and indexing functionality provided by Magento there exist a couple of (external) extensions that help you speed of content delivery by e.g. caching entire pages, such as PageCache for Varnish.

Simply put, Varnish represents a reverse-proxy cache that serves as entry-point into you Magento installation and returns cached content rather than requesting Magento to re-build entire pages.

The following steps show you can setup Varnish cache server for Magento 1.9 CE on Mac OSX.

First off, make sure you have Homebrew installed and configured properly. Based on Homebrew we can easily setup Varnish by issuing the following command:

bash-3.2$ brew install varnish
==> Installing varnish dependency: pkg-config
==> Downloading http://pkgconfig.freedesktop.org/releases/pkg-config-0.28.tar.gz
######################################################################## 100,0%
==> ./configure --prefix=/usr/local/Cellar/pkg-config/0.28 --disable-host-tool --with-internal-glib --with-pc-path=/usr/local/lib/p
==> make
==> make check
==> make install
  /usr/local/Cellar/pkg-config/0.28: 10 files, 600K, built in 97 seconds
==> Installing varnish dependency: pcre
==> Downloading ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/pcre-8.33.tar.bz2
######################################################################## 100,0%
==> ./configure --prefix=/usr/local/Cellar/pcre/8.33 --enable-utf8 --enable-unicode-properties --enable-pcregrep-libz --enable-pcregrep-libbz2 --
==> make
==> make test
==> make install
   /usr/local/Cellar/pcre/8.33: 140 files, 4,1M, built in 49 seconds
==> Installing varnish
==> Downloading http://repo.varnish-cache.org/source/varnish-3.0.3.tar.gz
######################################################################## 100,0%
==> ./configure --prefix=/usr/local/Cellar/varnish/3.0.3 --localstatedir=/usr/local/var
==> make install
==> Caveats
To have launchd start varnish at login:
    ln -sfv /usr/local/opt/varnish/*.plist ~/Library/LaunchAgents
Then to load varnish now:
    launchctl load ~/Library/LaunchAgents/homebrew.mxcl.varnish.plist
Warning: /usr/local/sbin is not in your PATH
You can amend this by altering your ~/.bashrc file
==> Summary
 /usr/local/Cellar/varnish/3.0.3: 51 files, 1,8M, built in 43 seconds

As shown in Brew’s output you can start Varnish by issuing the following command:

==> Caveats
To have launchd start varnish at login:
    ln -sfv /usr/local/opt/varnish/*.plist ~/Library/LaunchAgents
Then to load varnish now:
    launchctl load ~/Library/LaunchAgents/homebrew.mxcl.varnish.plist

In case the following error occurs:

bash-3.2$ sudo launchctl load ~/Library/LaunchAgents/homebrew.mxcl.varnish.plist"
launchctl: Couldn't" stat("~/Library/LaunchAgents/homebrew.mxcl.varnish.plist"): No such file or directory nothing found to load

try a re-install first (and additionally do a brew-update/updgrade):

bash-3.2$ sudo brew uninstall varnish
Uninstalling /usr/local/Cellar/varnish/3.0.3...

After a brew update/upgrade Varnish 4.0.0 is now being installed:

bash-3.2$ sudo brew install varnish
==> Downloading https://downloads.sf.net/project/machomebrew/Bottles/varnish-4.0.0.mavericks.bottle.tar.gz
######################################################################## 100,0%
==> Pouring varnish-4.0.0.mavericks.bottle.tar.gz
==> Caveats
To have launchd start varnish at login:
    ln -sfv /usr/local/opt/varnish/*.plist ~/Library/LaunchAgents
Then to load varnish now:
    launchctl load ~/Library/LaunchAgents/homebrew.mxcl.varnish.plist
==> Summary
 /usr/local/Cellar/varnish/4.0.0: 93 files, 1,9M

Unfortunately, this does not fix the missing plist file problem every time.

Nevertheless, you can easily create the plist file yourself, as mentioned here https://github.com/Homebrew/homebrew/issues/15922:

Simply copy the contents for the plist file by calling brew cat varnish:

bash-3.2$ sudo brew cat varnish
require 'formula'

class Varnish < Formula
  homepage 'http://www.varnish-cache.org/'
  url 'https://repo.varnish-cache.org/source/varnish-4.0.0.tar.gz'
  sha1 '6ad4fcf42a505a748ae993cb6ed579159e023633'

  bottle do
    sha1 "190bba2cf0e521b489cbd64cc698ba769b18cccd" => :mavericks
    sha1 "fd12d468c2328179bd402f1592b65ef69d6f1baa" => :mountain_lion
    sha1 "0e9717e705541a34906947eb7e49ab800204e2ff" => :lion
  end

  depends_on 'pkg-config' => :build
  depends_on 'pcre'

  resource "docutils" do
    url "https://pypi.python.org/packages/source/d/docutils/docutils-0.11.tar.gz"
    sha1 "3894ebcbcbf8aa54ce7c3d2c8f05460544912d67"
  end

  def install
    ENV.prepend_create_path "PYTHONPATH", buildpath+"lib/python2.7/site-packages"
    resource("docutils").stage do
      system "python", "setup.py", "install", "--prefix=#{buildpath}"
    end

    system "./configure", "--disable-dependency-tracking",
                          "--prefix=#{prefix}",
                          "--localstatedir=#{var}",
                          "--with-rst2man=#{buildpath}/bin/rst2man.py",
                          "--with-rst2html=#{buildpath}/bin/rst2html.py"
    system "make install"
    (var+'varnish').mkpath
  end

  test do
    system "#{opt_sbin}/varnishd", "-V"
  end

  def plist; <<-EOS.undent
      <?xml version="1.0" encoding="UTF-8"?>
      <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
      <plist version="1.0">
      <dict>
        <key>Label</key>
        <string>#{plist_name}</string>
        <key>ProgramArguments</key>
        <array>
          <string>#{opt_sbin}/varnishd</string>
          <string>-n</string>
          <string>#{var}/varnish</string>
          <string>-f</string>
          <string>#{etc}/varnish/default.vcl</string>
          <string>-s</string>
          <string>malloc,1G</string>
          <string>-T</string>
          <string>127.0.0.1:2000</string>
          <string>-a</string>
          <string>0.0.0.0:80</string>
        </array>
        <key>KeepAlive</key>
        <true/>
        <key>RunAtLoad</key>
        <true/>
        <key>WorkingDirectory</key>
        <string>#{HOMEBREW_PREFIX}</string>
        <key>StandardErrorPath</key>
        <string>#{var}/varnish/varnish.log</string>
        <key>StandardOutPath</key>
        <string>#{var}/varnish/varnish.log</string>
      </dict>
      </plist>
    EOS
  end
end

The plist content in this case is:

<?xml version="1.0" encoding="UTF-8"?>
      <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
      <plist version="1.0">
      <dict>
        <key>Label</key>
        <string>#{plist_name}</string>
        <key>ProgramArguments</key>
        <array>
          <string>#{opt_sbin}/varnishd</string>
          <string>-n</string>
          <string>#{var}/varnish</string>
          <string>-f</string>
          <string>#{etc}/varnish/default.vcl</string>
          <string>-s</string>
          <string>malloc,1G</string>
          <string>-T</string>
          <string>127.0.0.1:2000</string>
          <string>-a</string>
          <string>0.0.0.0:80</string>
        </array>
        <key>KeepAlive</key>
        <true/>
        <key>RunAtLoad</key>
        <true/>
        <key>WorkingDirectory</key>
        <string>#{HOMEBREW_PREFIX}</string>
        <key>StandardErrorPath</key>
        <string>#{var}/varnish/varnish.log</string>
        <key>StandardOutPath</key>
        <string>#{var}/varnish/varnish.log</string>
      </dict>
      </plist>

Now starting Varnish via launchctl succeeds:

bash-3.2$ sudo launchctl load ~/Library/LaunchAgents/homebrew.mxcl.varnish.plist

Time to test the service

Open a browser and navigate to 127.0.0.1:8080 (which is the port Varnish is listening to set by the configuration in the plist file from above) and check out the headers:

Accept-Ranges	bytes, bytes
Age	0
Connection	keep-alive
Content-Language	de
Content-Length	1087
Content-Type	text/html; charset=utf-8
Date	Fri, 30 May 2014 07:31:12 GMT
Server	Apache/2.4.4 (Unix) PHP/5.4.7 OpenSSL/1.0.1e
Vary	accept-language,accept-charset
Via	1.1 varnish
X-Varnish	1661470811

Congratulations, as you can see Varnish is working properly!

The next step is to configure Apache to listen to port 8080 (or any other port apart from 80) and Varnish on port 80.

That’s it 🙂

Custom build script for GIT based projects

When managing smaller GIT based projects I find it helpful to opt for custom build scripts rather than having to deal with the overhead of setting up proper Jenkins (or-likewise) runtime environments.

In order to deal with this need I’ve created a simple build script that creates a compressed TAR file based on committed files.

Furthermore, since I use version numbers for e.g. JavaScript builds it’s always nice to have automatic file inclusion/exlusion based on this number, say, read from a config file.

In this particular example I want to read the version number from a variable called MY_VERSION in file config/base.php and include a special JavaScript build-folder in the target compressed TAR file. In addition, I only want to include files in the src-folder, as well as manually include other folders.

So, here it is:

#!/bin/sh
#
# REQUIREMENTS:
# - git
# - php
# - sed
# - xargs
# - tar
#

#(absolute) path to PHP, or simply 'php' if it is in the PATH
PHP_PATH="php"

#release number should be read from config/base.php
PHP_REQUIRE="require('$(pwd)/config/base.php');"
RELEASE_NUMBER=`$PHP_PATH -r "$PHP_REQUIRE echo MY_VERSION;"`

#hold list of files to be added to release tar.gz
TMP_SRC_FILE_LIST=$(pwd)/TMP_RELEASE_INCLUDE

#get list of git's tracked files
git ls-files > $TMP_SRC_FILE_LIST

#filter only SRC directory
sed -i '/^src./!d' $TMP_SRC_FILE_LIST

#manually include additional folders of your choice
sed -i '/^src\/js\/folder1/d' $TMP_SRC_FILE_LIST
sed -i '/^src\/js\/folder2/d' $TMP_SRC_FILE_LIST

#add RELEASE directory only
`echo "src/js/release/$RELEASE_NUMBER" >> $TMP_SRC_FILE_LIST`

# enclose lines in "" to enable spaces in file and folder names
sed -i 's/^/"/g' $TMP_SRC_FILE_LIST
sed -i 's/$/"/g' $TMP_SRC_FILE_LIST

#create compresse TAR
cat $TMP_SRC_FILE_LIST | xargs tar czf release_$RELEASE_NUMBER.tar.gz

#clean up
rm $TMP_SRC_FILE_LIST

That’s it – Feel free to adjust it to your needs.

Managing Doctrine 2 Entity Model mapping classes using YamlDriver in Zend Framework 2

This post continues my approach to integrate Doctrine 2 into Zend Framework 2 by additionally setting up YamlDriver to generate and manage Doctrine 2 Entity model mapping files.

First, we need to install the YAML dependency symfony/yaml:

//...
"require": {
        "php": ">=5.3.3",
        "zendframework/zendframework": "2.3.*",
        "zendframework/zend-developer-tools": "dev-master",
        "doctrine/doctrine-module": "dev-master",
        "doctrine/doctrine-orm-module": "dev-master",
        "symfony/yaml": "dev-master"
    }

And rebuild our project:

$ php composer.phar update
Loading composer repositories with package information
Updating dependencies (including require-dev)
  - Installing symfony/yaml (dev-master cee3067)
    Cloning cee3067d680232674c6505d91db9d3d635a9a8f4

Writing lock file
Generating autoload files

In case you forget the install the symfony/yaml module you will get an error like:

Fatal error: Class ‘Symfony\Component\Yaml\Yaml’ not found in \vendor\doctrine\orm\lib\Doctrine\ORM\Mapping\Driver\YamlDriver.php on line 712

Next, activate the YamlDriver for our skeleton Application module:

'doctrine' => array(
        'driver' => array(
            /* => replace with YamlDriver below
              'application_entities' => array(
              'class' => 'Doctrine\ORM\Mapping\Driver\AnnotationDriver',
              'cache' => 'array',
              'paths' => array(__DIR__ . '/../src/Application/Entity')
              ),
              'orm_default' => array(
              'drivers' => array(
              'Application\Entity' => 'application_entities'
              )
              ) */

            'ApplicationYamlDriver' => array(
                'class' => 'Doctrine\ORM\Mapping\Driver\YamlDriver',
                'cache' => 'array',
                'extension' => '.dcm.yml',
                'paths' => array(__DIR__ . '/yml')
            ),
            'orm_default' => array(
                'drivers' => array(
                    'Application\Entity' => 'ApplicationYamlDriver',
                )
            )
        )),

Make sure to create the folder src/Application/config/yml in which we will put our YAML model files. You can change this folder to whatever you like but I think config is quite a safe place for model declaration files.

Trying to refresh our application at this point will generate a ReflectionException since our models are currently not in-sync.

Thus, let’s create our User Entity YAML file Application.Entity.User.dcm.yml:

Application\Entity\User:
  type: entity
  table: user
  id:
    id:
      type: integer
      generator:
        strategy: AUTO
  fields:
    name:
      type: string
      length: 50

Please note that you need to follow the naming convention of YAML files exactly:

MODULE.Entity.MODEL.dcm.yml

Now it’s time to generate our Entity class based on the YAML file:

$ vendor/bin/doctrine-module.bat orm:generate-entities module/Application/src/
Processing entity "Application\Entity\User"

Entity classes generated to "\module\Application\src"

You should now have a User.php class in src/Application/Entity:

<?php

namespace Application\Entity;

use Doctrine\ORM\Mapping as ORM;

/**
 * User
 */
class User
{
    /**
     * @var integer
     */
    private $id;

    /**
     * @var string
     */
    private $name;


    /**
     * Get id
     *
     * @return integer 
     */
    public function getId()
    {
        return $this->id;
    }

    /**
     * Set name
     *
     * @param string $name
     * @return User
     */
    public function setName($name)
    {
        $this->name = $name;

        return $this;
    }

    /**
     * Get name
     *
     * @return string 
     */
    public function getName()
    {
        return $this->name;
    }
}

Let’s quickly check this newly generates Entity by calling it in our IndexController:

public function indexAction() {
        $objectManager = $this
                ->getServiceLocator()
                ->get('Doctrine\ORM\EntityManager');

        $user = new \Application\Entity\User();
        $user->setName('Some great YAML developer :)');

        $objectManager->persist($user);
        $objectManager->flush();

        echo 'Hello there ' . $user->getId();
    }

As you can see I’ve used setName instead of setFullName from Zend Framework 2 and Doctrine 2 ORM Integration. Thus, reloading your application will throw an Exception:

An exception occurred while executing ‘INSERT INTO user (name) VALUES (?)’ with params [“Some great YAML developer :)”]:

SQLSTATE[42S22]: Column not found: 1054 Unknown column ‘name’ in ‘field list’

Our old database still has a field fullName instead of name. Thus, we first need to sync our database table user with our model. Since we are in a development environment we could check with a validate-schema call:

$ vendor/bin/doctrine-module.bat orm:validate-schema
[Mapping]  OK - The mapping files are correct.
[Database] FAIL - The database schema is not in sync with the current mapping file.

As you can see our database needs to be updated with our local model Entity mapping changes from our YAML file. So, let’s update it:

$ vendor/bin/doctrine-module.bat orm:schema-tool:update
ATTENTION: This operation should not be executed in a production environment.
           Use the incremental update to detect changes during development and use
           the SQL DDL provided to manually update your database in production.

The Schema-Tool would execute "1" queries to update the database.
Please run the operation by passing one - or both - of the following options:
    orm:schema-tool:update --force to execute the command
    orm:schema-tool:update --dump-sql to dump the SQL statements to the screen

As you can see Doctrine warns you that you shouldn’t call the update command in a production environment as it could lead to possible data loss. So, it’s always better to let it calculate the incremental update statements first to let you double check:

$ vendor/bin/doctrine-module.bat orm:schema-tool:update --dump-sql
ALTER TABLE user ADD name VARCHAR(50) NOT NULL, DROP fullName;

This statement shows what we’ve already discovered before – that we need to replace the column fullName with name.

So, go ahead and run this SQL statement to update your database.

Since I’m lazy this is just a proof a concept setup I will simply force-update the database via Doctrine (do not do this in a production environment!!):

$ vendor/bin/doctrine-module.bat orm:schema-tool:update --force
Updating database schema...
Database schema updated successfully! "1" queries were executed

Finally, our database is in-sync with our local mapping files:

$ vendor/bin/doctrine-module.bat orm:validate-schema
[Mapping]  OK - The mapping files are correct.
[Database] OK - The database schema is in sync with the mapping files.

Now that we have a sync’ed Entity mapping class let’s try to refresh our application again:

Success! The DBAL Exception is gone and we are actively using our Entity model mapping class generated from our Yaml file.

At this point you might ask what happens to generated Entity mapping class files when your YAML files change. Well, when calling generate-entities on changed YAML files Doctrine will try to merge custom code with generated stubs of the getter/setter methods. Be sure to backup your generated mapping classes before calling generate-entities to avoid loss of your precious code as this merge will definitely not work at all times.

Zend Framework 2 and Doctrine 2 ORM Integration

So, finally I’ve found time to start a new approach of migrating older Zend Framework (ZF) 1 projects to ZF2. To get most out of ZF2 and its new modular design I’ve decided to simultaneously try out the Doctrine 2 ORM module. These are the steps I’ve used to integrate Doctrine 2 into ZF2.

Skeleton ZF2 Project

First off, setup your skeleton ZF2 project as usual. Using Netbeans (or any IDE of your choice) download the ZendSkeletonApplication and create a new project. You will also need to setup composer to manage your ZF2 project – so do so as well.

Then, initialize your skeleton application by issuing the install command:

$ php composer.phar install
Warning: This development build of composer is over 30 days old. It is recommended to update it by running &amp;quot;composer.phar self-update&amp;quot; to get the latest version              .
Loading composer repositories with package information
Installing dependencies (including require-dev)
  - Installing zendframework/zendxml (1.0.0)
    Downloading: 100%

  - Installing zendframework/zendframework (2.3.0)
    Downloading: 100%

zendframework/zendframework suggests installing ext-intl (ext/intl for i18n features (included in default builds of PHP))
zendframework/zendframework suggests installing doctrine/annotations (Doctrine Annotations &amp;gt;=1.0 for annotation features)
zendframework/zendframework suggests installing ircmaxell/random-lib (Fallback random byte generator for ZendMathRand if OpenSSL/Mcrypt extensions are unavailable)
zendframework/zendframework suggests installing ocramius/proxy-manager (ProxyManager 0.5.* to handle lazy initialization of services)
zendframework/zendframework suggests installing zendframework/zendpdf (ZendPdf for creating PDF representations of barcodes)
zendframework/zendframework suggests installing zendframework/zendservice-recaptcha (ZendServiceReCaptcha for rendering ReCaptchas in ZendCaptcha and/or ZendForm)
Writing lock file
Generating autoload files

This sets up your dev environment by downloading ZF2 into your project’s vendor folder. Based on the shiny new modular design composer also suggests to install additional modules, such as ext-intl and doctrine-annotations. We ignore theses suggestions for now and continue to install the Doctrine ORM module first.

Creating ZF2 Modules via command line

You can use the Zend Framework Tool (ZFTool) to create ZF2 modules from the command line too, just in case you want to start from scratch. Setup ZFTool via composer (zendframework/zftool) and run the following command to create a ZF2 module:

zf.php.bat create YourModule

Make sure to run zf.php(.bat) from the root of your project – otherwise you’ll get an error saying

I cannot find the autoloader of the application.

Additional Modules

In order to install additional modules you basically have two options with composer:

  1. Add required modules in the require-section in composer.json
  2. Install module(s) via composer from command line

I tend to choose the first option. Thus, we need to add the following lines to the require section in composer.json:

{
    &amp;quot;require&amp;quot;: {
        &amp;quot;php&amp;quot;: &amp;quot;&amp;gt;=5.3.3&amp;quot;,
        &amp;quot;zendframework/zendframework&amp;quot;: &amp;quot;2.3.*&amp;quot;,
        &amp;quot;doctrine/doctrine-module&amp;quot;: &amp;quot;dev-master&amp;quot;,
        &amp;quot;doctrine/doctrine-orm-module&amp;quot;: &amp;quot;dev-master&amp;quot;
    }
}

Since doctrine-orm-module requires doctrine-module we include it explicitely. Optionally, you could use specific vesions for both modules but since this is a plain dev project dev-master is just fine. Make sure to use specific versions for production environments!

To update your ZF2 project with these new requirements simply call composer with the update switch:

$ php composer.phar update
Loading composer repositories with package information
Updating dependencies (including require-dev)
  - Installing symfony/console (v2.4.3)
    Downloading: 100%

  - Installing doctrine/lexer (v1.0)
    Downloading: 100%

  - Installing doctrine/annotations (v1.1.2)
    Downloading: 100%

  - Installing doctrine/collections (v1.2)
    Downloading: 100%

  - Installing doctrine/cache (v1.3.0)
    Downloading: 100%

  - Installing doctrine/inflector (v1.0)
    Downloading: 100%

  - Installing doctrine/common (v2.4.1)
    Downloading: 100%

  - Installing doctrine/doctrine-module (dev-master 3af3a00)
    Cloning 3af3a000644eeee27e76f53ece644333caf957e8

 - Installing doctrine/dbal (v2.4.2)
    Downloading: 100%

  - Installing doctrine/orm (v2.4.2)
    Downloading: 100%

  - Installing doctrine/doctrine-orm-module (dev-master 0aac7bd)
    Cloning 0aac7bd1a9948418156178977c21cfd4d93aee41

Writing lock file
Generating autoload files

We have successfully added the Doctrine 2 ORM module to our ZF2 project – congratulations 🙂

Zend Developer Tools

Optionally, you can also install the Zend Developer Tools module which is highly recommended at this point:

{
    &amp;quot;require&amp;quot;: {
        &amp;quot;php&amp;quot;: &amp;quot;&amp;gt;=5.3.3&amp;quot;,
        &amp;quot;zendframework/zendframework&amp;quot;: &amp;quot;2.3.*&amp;quot;,
        &amp;quot;zendframework/zend-developer-tools&amp;quot;: &amp;quot;dev-master&amp;quot;,
        &amp;quot;doctrine/doctrine-module&amp;quot;: &amp;quot;dev-master&amp;quot;,
        &amp;quot;doctrine/doctrine-orm-module&amp;quot;: &amp;quot;dev-master&amp;quot;
    }
}

… and update your project with the new dependency:

$ php composer.phar update
Loading composer repositories with package information
Updating dependencies (including require-dev)
  - Installing zendframework/zend-developer-tools (dev-master 763cecf)
    Cloning 763cecf9d68fc0d3765d45847047817a47ecdd4a

Writing lock file
Generating autoload files

In order to use Zend Developer Tools with need to auto-load its configuration. Thus, copy the module’s dist file to your application’s autoload folder and remove the .dist extension to make it a regular PHP file:

cp vendor/zendframework/zend-developer-tools/config/zenddevelopertools.local.php.dist config/autoload/zenddevelopertools.local.php

Also, add this module to the application.config.php:

&amp;lt;?php
  return array(
    // This should be an array of module namespaces used in the application.
    'modules' =&amp;gt; array(
        'Application',
        'ZendDeveloperTools',
        ...
    ),

To test the developer tools simply refresh your skeleton application. At the bottom you should now see some development output, as shown below:

zdt-zf2-toolbar

Setup Doctrine 2

Next, we need to actually setup our shiny new Doctrine 2 module.

First, activate it by updating your config/application.config.php:

return array(
  'modules' =&amp;gt; array(
        'ZendDeveloperTools',
        'DoctrineModule',
        'DoctrineORMModule',
        'Application',
    ),
    // more config...

Now we need to configure the database connection so let’s create a config/autoload/doctrine.local.php with our credentials:

&amp;lt;?php

$dbParams = array(
    'hostname' =&amp;gt; 'localhost',
    'port' =&amp;gt; 3306,
    'username' =&amp;gt; 'root',
    'password' =&amp;gt; '',
    'database' =&amp;gt; 'zf2doctrine2'
);

return array(
    'doctrine' =&amp;gt; array(
        'connection' =&amp;gt; array(
            'orm_default' =&amp;gt; array(
                'params' =&amp;gt; array(
                    'host' =&amp;gt; $dbParams['hostname'],
                    'port' =&amp;gt; $dbParams['port'],
                    'user' =&amp;gt; $dbParams['username'],
                    'password' =&amp;gt; $dbParams['password'],
                    'dbname' =&amp;gt; $dbParams['database'],
                    'driverOptions' =&amp;gt; array(
                        PDO::MYSQL_ATTR_INIT_COMMAND =&amp;gt; 'SET NAMES utf8'
                    ),
                )
            )
        )
    )
);

Note that you can also simply put the database connection details in local.php and/or global.php depending on your environment (dev vs production) but I tend to seperate these configuration options into different files.

Optional: You can also setup query debugging tools, such as BjyProfiler. Like the Zend Developer Tools you can easily set it up via composer:

&amp;quot;bjyoungblood/BjyProfiler&amp;quot;: &amp;quot;dev-master&amp;quot;

and activate by configuring bjyprofiler.local.php in autoload.

Setting Annotation Driver

That’s it! Now you are ready to create your first Entity file:

To use the modules auto-sync funtionality we need to add support for annotation mappings in the corresponding module’s module.config.php:

'doctrine' =&amp;gt; array(
        'driver' =&amp;gt; array(
            'application_entities' =&amp;gt; array(
                'class' =&amp;gt; 'DoctrineORMMappingDriverAnnotationDriver',
                'cache' =&amp;gt; 'array',
                'paths' =&amp;gt; array(__DIR__ . '/../src/Application/Entity')
            ),
            'orm_default' =&amp;gt; array(
                'drivers' =&amp;gt; array(
                    'ApplicationEntity' =&amp;gt; 'application_entities'
                )
            ))),

As specified by the paths option make sure to put Entity files in src/Application/Entity for the Application module.

Now let’s create our first entity file:

&amp;lt;?php

namespace ApplicationEntity;

use DoctrineORMMapping as ORM;

/** @ORMEntity */
class User {

    /**
     * @ORMId
     * @ORMGeneratedValue(strategy=&amp;quot;AUTO&amp;quot;)
     * @ORMColumn(type=&amp;quot;integer&amp;quot;)
     */
    protected $id;

    /** @ORMColumn(type=&amp;quot;string&amp;quot;) */
    protected $fullName;

    /**
     * Get id
     *
     * @return integer
     */
    public function getId() {
        return $this-&amp;gt;id;
    }

    /**
     * Set fullname
     *
     * @param string $fullname
     * @return User
     */
    public function setFullname($fullname) {
        $this-&amp;gt;fullName = $fullname;

        return $this;
    }

}

Upon reload of your application you will notice 1 mapping:

zdt-zf2-toolbar-mapping

As you can see the dynamic mapping already works!

Let’s validate the database sync status:

$ vendor/bin/doctrine-module.bat orm:validate-schema
[Mapping]  OK - The mapping files are correct.
[Database] FAIL - The database schema is not in sync with the current mapping file.

Obviously, the local entities are not in snc with our database. So let’s sync it:

$ vendor/bin/doctrine-module.bat orm:schema-tool:create
ATTENTION: This operation should not be executed in a production environment.

Creating database schema...
Database schema created successfully!

And finally test it:

public function indexAction() {
    $objectManager = $this
        -&amp;gt;getServiceLocator()
        -&amp;gt;get('DoctrineORMEntityManager');

    $user = new ApplicationEntityUser();
    $user-&amp;gt;setFullName('Some great developer :)');

    $objectManager-&amp;gt;persist($user);
    $objectManager-&amp;gt;flush();

    die('Hello there '.$user-&amp;gt;getId());
}

Check the debug tools:

zdt-zf2-toolbar-debugger

That’s it!

Want to know how to generate Entity model mapping classes at ease based on YAML files? Then read on: Using YamlDriver to generate Entity model mapping classes via YAML files.