Tag: 201

  • Configure robots.txt for Realurl in TYPO3

    Configure robots.txt for Realurl in TYPO3

    In order to configure robots.txt for the Realurl extension in TYPO3 you need to set two things:

    1. Add filename for page type 201 in realurl_config.php
    2. Add some TypoScript to process robots generation

    Add filename for page type in realurl_config.php

    $TYPO3_CONF_VARS['EXTCONF']['realurl']['_DEFAULT'] = array(
      ...
      // configure filenames for different pagetypes
        'fileName' => array(
            'defaultToHTMLsuffixOnPrev' => 0,
            'index' => array(
                'print.html' => array(
                    'keyValues' => array(
                        'type' => 98,
                    ),
                ),
                ...
                // add robots.txt page type
                'robots.txt' => array(
                    'keyValues' => array(
                        'type' => 201
                    )
                )
            ),
      ...
    

    Add TypoScript to process robots.txt generation

    robots = PAGE
    robots {
      typeNum = 201
      config {
        disableAllHeaderCode = 1
        additionalHeaders = Content-type:text/plain
      }
      10 = TEXT
      10 {
        wrap (
    User-Agent: *
    Allow: /                #allow everything by default
    Disallow: /fileadmin/templates     #then add restrictions
    Disallow: /typo3/        
    Disallow: /t3lib/        
    Disallow: /typo3conf/    
    Disallow: /typo3temp/    
    Disallow: /*?id=*        
    
    User-agent: googlebot  # Google specific settings
    Disallow: /*?tx_indexedsearch
    
    Sitemap: /?eID=dd_googlesitemap # finally add some sitemap
        )
    } 
    

    Be sure to flush the cache and you are all set!