skip to content
Advertisement

Desert Realm Spider

Name Desert Realm Spider
Cover http://www.desertrealm.com
Details http://spider.desertrealm.com
Status robot actively in use
Description The spider indexes fantasy and science fiction sites by using a customizable keyword algorithm. Only home pages are indexed, but all pages are looked at for links. Pages are visited randomly to limit impact on any one webserver.
Purpose indexing
Type standalone
Platform cross platform
Language java 1.3, java 1.4
Availability none
Owner Name Brian B.
Owner URL http://www.desertrealm.com
Owner Email spider@desertrealm.com
Exclusion yes
Exclusion User-Agent desertrealm, desert realm
NOINDEX yes
Host *
From no
UserAgent DesertRealm.com; 0.2; [J];
History The spider originally was created to learn more about how search engines work.
Environment hobby
ID desertrealm
Modified Date Fri, 19 Sep 2003 08:57:52 GMT
Modified By Brian B.
Advertisement