User-agent: ilial Disallow: / User-agent: VSynCrawler Disallow: / User-agent: CazoodleBot Disallow: / User-agent: shelob Disallow: / User-agent: shelob v1.0 Disallow: / User-agent: Shelob (shelob@gmx.net) Disallow: / User-agent: SBIder/SBIder-0.8.2-dev Disallow: / User-agent: SBIder Disallow: / User-agent: Spock Crawler Disallow: / User-agent: BrightCrawler Disallow: / User-agent: Snoopy v1.2 Disallow: / User-Agent: OmniExplorer_Bot Disallow: / User-Agent: Gigamega.bot Disallow: / User-Agent: Gigamega.bot/1.0 Disallow: / User-Agent: VadixBot Disallow: / User-Agent: Krugle/Krugle,Nutch/0.8+ Disallow: / User-Agent: Krugle/Krugle Disallow: / User-Agent: Krugle Disallow: / User-Agent: McBot/5.001 Disallow: / User-Agent: disco/Nutch-1.0-dev Disallow: / User-Agent: disco Disallow: / User-Agent: SpokeSpider/1.0 Disallow: / User-agent: Teemer Disallow: / User-agent: Yeti/0.01 Disallow: / User-agent: Mail.Ru/1.0 Disallow: / User-agent: McBot/5.001 Disallow: / User-agent: McBot Disallow: / User-agent: panscient.com Disallow: / User-agent: MLBot Disallow: / User-agent: WebAlta Crawler/2.0 Disallow: / User-agent: EmailSearch Disallow: / User-agent: Gigabot/3.0 Disallow: / User-agent: AISearchBot Disallow: / User-agent: DataCha0s/2.0 Disallow: / User-agent: core-project/1.0 Disallow: / User-agent: 008 Disallow: /