################# # Robot.txt file # set for all Robots (yahoo, google, msn, etc...) # delay at least 3 seconds # disallow larger directories that should not be indexed # User-agent: * Crawl-delay: 3 Disallow: /email/ Disallow: /mdbe/ Disallow: /processes/