# No web spiders allowed in these directories User-agent: * Disallow: /sei Disallow: /Courses2006 # Stop IE's offline-content from trying to get to this stuff User-Agent:MSIECrawler Disallow:/flc/staff_use/ User-agent:Googlebot Disallow:/resources/09915/