skip to content

Other Sites


Many people end up on this site because they have questions about specific search engine robots and search engines. For such questions the best place is the relevant's site's own help pages:

Extensions to the Robots Exclusion Protocol

Recently three major search engines have collaborated to support extensions to the /robotst.txt directives and related mechanisms. See the join announcements on:

Sites about Search Engines

Other sites that are useful to webmasters: