#
# robots.txt
#
# This file is to prevent the crawling and indexing of certain parts
# of your site by web crawlers and spiders run by sites like Yahoo!
# and Google. By telling these robots where not to go on your site,
# you save bandwidth and server resources.
#
# This file will be ignored unless it is at the root of your host
# Used httpexample.comrobots.txt
# Ignored httpexample.comsiterobots.txt
#
# For more information about the robots.txt standard, see
# httpwww.robotstxt.orgwcrobots.html
#
# For syntax checking, see
# httpwww.sxw.org.ukcomputingrobotscheck.html
User-agent: *
# Directories
Disallow: includes
Disallow: misc
Disallow: modules
Disallow: profiles
Disallow: scripts
Disallow: themes
# Files
Disallow: CHANGELOG.txt
Disallow: cron.php
Disallow: INSTALL.mysql.txt
Disallow: INSTALL.pgsql.txt
Disallow: INSTALL.sqlite.txt
Disallow: install.php
Disallow: INSTALL.txt
Disallow: LICENSE.txt
Disallow: MAINTAINERS.txt
Disallow: update.php
Disallow: UPGRADE.txt
Disallow: xmlrpc.php
Google 
Alexa 
Bing 
Aol 
Search.com