Magento Robots.txt File


The robots.txt file on your website can help you say what gets indexed, crawled and used by the search engines. It is a vital part of your website which is often overlooked. Full details on the robots.txt file can be found on the wiki page @ http://en.wikipedia.org/wiki/Robots_exclusion_standard however below I have listed the most helpful robots.txt for a Magneto website:

Simply copy the above into a text file called robots.txt and upload it into the base of your Magento installation (e.g. where index.php is).

Unknown source


Leave a reply