# see http://www.searchengineworld.com/robots/ for tutorial and examples # how to specifiy robots (spiders) and items to disallow # keep everyone out of the sandbox and image trees User-agent: * Disallow: /images/ Disallow: /sandbox/ Disallow: /dir1/image/ Disallow: /dir1/Catskills/images/ Disallow: /dir2/image/ Disallow: /dir2/Catskills/images/ # restrict SMF crawling to login/registration Disallow: /forum1/Themes/ Disallow: /forum1/Sources/ Disallow: /forum1/*.msg* Disallow: /forum1/*sa=showPosts* Disallow: /forum1/*prev_next* Disallow: /forum1/*action=emailuser* Disallow: /forum1/*action=printpage* Disallow: /forum1/*action=recent* Disallow: /forum1/*action=help* Disallow: /forum1/*action=login* Disallow: /forum1/*action=profile* Disallow: /forum1/*action=register* Disallow: /forum1/*action=search* Disallow: /forum1/*action=stats* Disallow: /forum1/*action=unread* Disallow: /forum1/*action=verificationcode* Disallow: /forum1/*action=who* #sitemap: http://www.catskilltech.com/sitemap.xml