# /robots.txt for www.blastwave.org
#--------------------------------------------------------------------------
# The following is a comment located in the robots.txt file at www.sun.com
# which I preserved
# Dennis Clarke  dclarke@blastwave.org
#--------------------------------------------------------------------------
# Mon Feb  2 11:59:27 PST 1998, Fred Elliott
# A NOTE TO THOSE WHO'D BOTHER TO LOOK AT THIS FILE:
#
# Bertrand Meyer's excellent "comp.risks" posting about the potential
# for misusing "robots.txt" files
# (http://www.eiffel.com/private/meyer/robots.html) includes a snapshot
# of the contents of this file here on www.sun.com.
#
# In the article, Bertrand speculates that the directories listed below
# contain proprietary information.  Well, they don't.  They do, though,
# contain information that we'd prefer people register for before they
# download it.
#
# The purpose of the "robots.txt" file is to keep these directories
# from being indexed so that the average user doesn't stumble across them
# while performing searches, and those that should be accessing these
# directories will do so through the URL that requires them to register.
# Of course, having the contents of this file advertised in "comp.risks"
# diminishes its purpose.  Thanks Bertrand. ;-)
#
# If you do actually go to the trouble of figuring out how to download
# the files without registering, what you'll end up with is 1 or 2MB of
# stuff that is meaningless to you unless you have purchased an
# Ultra AX board from Sun.  So, please do purchase an Ultra AX board,
# but then you might as well use the URL you'll be given along with it.
#--------------------------------------------------------------------------
#

User-agent: *

Disallow: /sitestats

User-agent: Slurp
Disallow: /

User-agent: msnbot
Disallow: /

User-agent: Teoma
Disallow: /

User-agent: http://www.almaden.ibm.com/cs/crawler 
Disallow: /

User-agent: BecomeBot
Disallow: /

User-agent: Shopwiki
Disallow: /

User-agent: Accoona-AI-Agent
Disallow: /

User-agent: Tasapspider
Disallow: /

User-agent: Exabot
Disallow: /

User-agent: dotbot
Disallow: /