User-agent: * Crawl-delay: 60 Request-rate: 1/60 # maximum rate is one page every 60 seconds Visit-time: 2330-0530 # only visit between 00:30 and 06:30 CET (parameter is in UTC (GMT)) Disallow: /SomeDisallowedDir/