How to generate a robots.txt file in Prestashop 1.5
It is often a good idea to restrict search engine spiders or crawlers from accessing certain parts of your website. This is done with a file called a ROBOTS.TXT file. Fortunately, Prestashop 1.5 does this for you. The following tutorial will walk you through the process of getting the file created.
How to create a robots.txt file in Prestashop 1.5
- Login to the Administration Panel as an Administrator
- Hover over the menu bar where it says PREFERENCES, and then click on SEO & URLS in the drop down menu. You will see an existing list of URLs. Scroll down further until you see ROBOTS FILE GENERATION. The following screenshot indicates the button to click that will generate the ROBOTS.TXT file:
- The information outlined in blue is important as it lets you know where the file is required to be located. By default, in Web Host Hub's servers, the location of the file is in the PUBLIC_HTML folder if the website is the primary domain. If it is an addon domain, then the path to the ROBOTS file will look like this:
/public_html/add-on-domain-name.com
If you are not hosting with Web Hosting Hub, please consult your Technical Support or documentation about the location of your files. - Click on the GENERATE ROBOTS.TXT FILE button and the ROBOTS file will automatcially be created.
- The ROBOTS.TXT file will generate the necessary information to limit access to search engine spiders and crawlers. Here's the top of the file:
# robots.txt automaticaly generated by PrestaShop e-commerce open-source solution
# http://www.prestashop.com - http://www.prestashop.com/forums
# This file is to prevent the crawling and indexing of certain parts
# of your site by web crawlers and spiders run by sites like Yahoo!
# and Google. By telling these "robots" where not to go on your site,
# you save bandwidth and server resources.
# For more information about the robots.txt standard, see:
# http://www.robotstxt.org/wc/robots.html
The rest of the file includes the entries to keep the robot from going where it doesn't need to go. The most important part to take from this is that limiting the access helps to save BANDWIDTH and SERVER RESOURCES.
Did you find this article helpful?
Tweet
Comments
n/a Points
|
2014-06-06 11:41 pm
I´m working on Prestashop 1.6. Because I´m adding a shop to an existing page I have put prestashop into a subfolder = shop. I know the generated robots.txt file has to be moved to the root. But do I have to change the robots.txt lines in it to work from the root or does all these * in the file mean, that it will work from the root? From the generated file. # Private pages Disallow: /*controller=addresses
# Directories Disallow: */classes/Disallow: */config/ Kind regards Thomas |
Staff 17,314 Points
|
2014-06-07 2:18 am
Hello Thomas,
You just need to make sure that any specific paths are correct. The general references such as the ones you've listed above should work fine. Check out our tutorial called Using Robots.txt for more specific information on the commands. It also provides links for more detail information on the use of the robots.txt file. I hope that helps to answer your question. Please let us know if you require further assistance. Regards, Arnel C. |
|
n/a Points
|
2014-06-07 2:31 am
Hi Arnel. Thank you for your quick responce. ok, so the * simply is a message for all robots? The lines you saw in the first message is generated by prestashop.
So I will have to change the path to
# Private pages Disallow: /shop/controller=addresses (where to place the * )
# Directories (where to place the *) Disallow: /shop/classes/ Disallow: /shop/config/
Best Thomas |
|
Staff 17,314 Points
|
2014-06-07 2:35 am
Hello Thomas,
That is correct. The * indicates any robot (also called user agent). Regards, Arnel C. |
Help Center Login
Base Configuration and Preferences in PrestaShop 1.5
Related Questions
Here are a few questions related to this article that our customers have asked:
Ooops! It looks like there are no questions about this page.
Would you like to ask a question about this page? If so, click the button below!
We value your feedback!
There is a step or detail missing from the instructions.
The information is incorrect or out-of-date.
It does not resolve the question/problem I have.
new! - Enter your name and email address above and we will post your feedback in the comments on this page!