Tag: robots.txt

Sample SEO Magento robots.txt file

Since I get a lot of requests for a robots.txt file designed for Magento SEO here is a sample to get you started. This Magento robots.txt makes the following assumptions: We don’t differentiate between search engines, hence User-agent: * We allow assets to be crawled i.e. images, CSS and JavaScript files We only allow SEF URLs […]

SEO Key Metrics and Factors for implementing a SEO quick check tool

In case you are searching for ways on how to implement a simple but useful SEO quick check tool here are some tips on the key metrics and factors required to get you started. The SEO quick check tool itself is based on PHP to provide a proof of concept that can be easily adapted […]

Configure robots.txt for Realurl in TYPO3

In order to configure robots.txt for the Realurl extension in TYPO3 you need to set two things: Add filename for page type 201 in realurl_config.php Add some TypoScript to process robots generation Add filename for page type in realurl_config.php Add TypoScript to process robots.txt generation Be sure to flush the cache and you are all […]