User-Agent: MJ12bot
Request-rate: 6/1m
Crawl-delay: 10
User-agent: Baiduspider
Request-rate: 6/1m
Crawl-delay: 10
User-agent: spbot
Request-rate: 6/1m
Crawl-delay: 10
User-agent: WeSEE
Request-rate: 6/1m
Crawl-delay: 10
User-agent: grapeshot
Request-rate: 6/1m
Crawl-delay: 10
User-agent: linkdexbot
Request-rate: 6/1m
Crawl-delay: 10
User-agent: Qwantify
Request-rate: 6/1m
Crawl-delay: 10
User-agent: Abonti
Request-rate: 6/1m
Crawl-delay: 10
User-agent: Findxbot
Request-rate: 6/1m
Crawl-delay: 10
User-agent: BLEXBot
Request-rate: 6/1m
Crawl-delay: 10