############### robots.txt ############ ####################################### User-agent: * ####################################### ############### SITEMAP ############### ####################################### Sitemap: https://www.growell.co.uk/media/sitemap.xml Sitemap: https://www.growell.co.uk/media/sitemap_blog.xml Sitemap: https://www.growell.co.uk/media/sitemap_shops.xml ####################################### ################ PAGES ################ ####################################### Disallow: /privacy-policy-cookie-restriction-mode/ Disallow: /terms/ Disallow: /hidden.html Disallow: /ebay-only.html ####################################### ##### MAGENTO DIRECTORIES & FILES ##### ####################################### ##### Directories ##### Disallow: /404/ Disallow: /app/ Disallow: /cgi-bin/ Disallow: /downloader/ Disallow: /includes/ Disallow: /lib/ Disallow: /magento/ Disallow: /pkginfo/ Disallow: /report/ Disallow: /stats/ Disallow: /var/ ##### Paths (clean URLs) ##### Disallow: /index.php/ Disallow: /catalog/product_compare/ Disallow: /catalog/category/view/ Disallow: /catalog/product/view/ Disallow: /catalog/product/gallery/ Disallow: /catalogsearch/ Disallow: /checkout/ Disallow: /control/ Disallow: /contacts/ Disallow: /customer/ Disallow: /customize/ Disallow: /newsletter/ Disallow: /poll/ Disallow: /review/ Disallow: /sendfriend/ Disallow: /tag/ Disallow: /wishlist/ ##### Last updated 2016-09-08: to add new strings to "/?" Query Strings ##### ##### Updated 2017-05-29: to remove 'brand' from the exclusion list test ##### ## ## Disallow: /?Brand ## ## ## ## Disallow: /?brand ## ## Disallow: /?diameter Disallow: /?extraction_rate Disallow: /?filter_type Disallow: /?mode Disallow: /?price Disallow: /?dir Disallow: /?order Disallow: /?type2 Disallow: /?fan_type Disallow: /?kit_type Disallow: /?water_flow_rate Disallow: /?media_type Disallow: /?growing_method Disallow: /?number_of_plants Disallow: /?volume Disallow: /?ballast_type Disallow: /?light_size Disallow: /?lamp_type Disallow: /?primary_use Disallow: /?power_rating Disallow: /?max_width Disallow: /?number_of_lights Disallow: /?max_height Disallow: /?application Disallow: /?nutrient_n_booster_type Disallow: /?pest_problem Disallow: /?method Disallow: /?max_depth Disallow: /?tent_type Disallow: /?limit Disallow: /?tent_interior Disallow: /?rough_guide ##### Last updated 2017-02-24: to add new strings to "/?" Query Strings ##### Disallow: ?tab=reviews Disallow: /?tab=reviews ##### Added 2017-04-19: adding exclusions for old CMS structure where we have "?" query strings in the url ##### ##### Added updated 2017-04-19: also managing the reflector, limit and price variables on a few page requests ##### Disallow: /*.php3?*id= Disallow: /*.php?*f=register Disallow: /*.php?*f=baskcode Disallow: /*.php?*f=friend Disallow: /*.php?*f=search Disallow: /*.html?*tab= Disallow: reflectors? Disallow: *?limit Disallow: *price ##### Last updated 2016-11-02: ".html?" Query Strings to deal with .html file name ##### ##### Updated 2017-05-29: to remove 'brand' from the exclusion list ##### Disallow: /*.html?*application= Disallow: /*.html?*ballast_type= Disallow: /*.html?*controller_type= Disallow: /*.html?*diameter= Disallow: /*.html?*disease= Disallow: /*.html?*dir= Disallow: /*.html?*extraction_rate= Disallow: /*.html?*fan_size= Disallow: /*.html?*fan_type= Disallow: /*.html?*filter_type= Disallow: /*.html?*growing_method= Disallow: /*.html?*kit_type= Disallow: /*.html?*lamp_type= Disallow: /*.html?*light_size= Disallow: /*.html?*limit= Disallow: /*.html?*max_depth= Disallow: /*.html?*max_height= Disallow: /*.html?*max_width= Disallow: /*.html?*maximum_load= Disallow: /*.html?*media_type= Disallow: /*.html?*method= Disallow: /*.html?*mode= Disallow: /*.html?*number_of_lights= Disallow: /*.html?*number_of_plants= Disallow: /*.html?*nutrient_n_booster_type= Disallow: /*.html?*order= Disallow: /*.html?*pest_problem= Disallow: /*.html?*power_rating= Disallow: /*.html?*price= Disallow: /*.html?*primary_use= Disallow: /*.html?*refillable= Disallow: /*.html?*reflective_finish= Disallow: /*.html?*roll_width= Disallow: /*.html?*rough_guide= Disallow: /*.html?*temperature_control= Disallow: /*.html?*tent_interior= Disallow: /*.html?*tent_type= Disallow: /*.html?*type2= Disallow: /*.html?*voltage= Disallow: /*.html?*volume= Disallow: /*.html?*water_flow_rate= #### Last updated 2016-11-02: Prior spam urls are now returning 410, add these to the 420 file #### #### Last updated 2016-11-02: Addition of no index spam urls #### #### Last updated 2016-04-19: Addition of no index spam urls #### ##### Files ##### Disallow: /cron.php Disallow: /cron.sh Disallow: /error_log Disallow: /install.php Disallow: /LICENSE.html Disallow: /LICENSE.txt Disallow: /LICENSE_AFL.txt Disallow: /STATUS.txt ####################################### #### WORDPRESS DIRECTORIES & FILES #### ####################################### Disallow: /blog/wp-content/upgrade/ Disallow: /blog/wp-admin/ Disallow: /blog/wp-includes/ #### Last updated 2016-11-02: Addition of wp-system exclude #### Disallow: /wp-system/* #### Created 2018-01-23: No index old, spam pages #### #### Updated 2018-09-24: No index old, spam pages #### #### Updated 2019-09-11: Remove Noindex references #### ####################################### ############### robots.txt ############ #######################################