skip to content

Desert Realm Spider

Name Desert Realm Spider
Status robot actively in use
Description The spider indexes fantasy and science fiction sites by using a customizable keyword algorithm. Only home pages are indexed, but all pages are looked at for links. Pages are visited randomly to limit impact on any one webserver.
Purpose indexing
Type standalone
Platform cross platform
Language java 1.3, java 1.4
Availability none
Owner Name Brian B.
Owner URL
Owner Email
Exclusion yes
Exclusion User-Agent desertrealm, desert realm
Host *
From no
UserAgent; 0.2; [J];
History The spider originally was created to learn more about how search engines work.
Environment hobby
ID desertrealm
Modified Date Fri, 19 Sep 2003 08:57:52 GMT
Modified By Brian B.