skip to content

How do I know if I've been visited by a robot?

<< | >>

You can check your server logs for sites that retrieve many documents, especially in a short time.

If your server supports User-agent logging you can check for retrievals with unusual User-agent header values.

Finally, if you notice a site repeatedly checking for the file '/robots.txt' chances are that is a robot too.