# See http://www.robotstxt.org/wc/norobots.html for documentation on how to use the robots.txt file # # To ban all spiders from the entire site uncomment the next two lines: User-Agent: * Disallow: /admin Disallow: /sign_up Disallow: /sign_in Disallow: /sign_out Disallow: /passwords Disallow: /contact Disallow: /customer Disallow: /customer_sessions Disallow: /basket_items Disallow: /orders Disallow: /items # Disallow: /skus # Disallow: /categories/digital-multimedia/PDF Disallow: /lecturer_profiles Disallow: /lecturer Disallow: /courses