Harvest
| Name | Harvest |
|---|---|
| Cover | http://harvest.cs.colorado.edu |
| Details | |
| Status | |
| Description | Harvest's motivation is to index community- or topic- specific collections, rather than to locate and index all HTML objects that can be found. Also, Harvest allows users to control the enumeration several ways, including stop lists and depth and count limits. Therefore, Harvest provides a much more controlled way of indexing the Web than is typical of robots. Pauses 1 second between requests (by default). |
| Purpose | indexing |
| Type | |
| Platform | |
| Language | |
| Availability | |
| Owner Name | |
| Owner URL | |
| Owner Email | |
| Exclusion | |
| Exclusion User-Agent | |
| NOINDEX | |
| Host | bruno.cs.colorado.edu |
| From | yes |
| UserAgent | yes |
| History | |
| Environment | |
| ID | harvest |
| Modified Date | |
| Modified By |