# www.robotstxt.org/ # http://code.google.com/web/controlcrawlindex/ User-agent: * Disallow: /framework/ Disallow: /application/ Disallow: /archives/ Disallow: /dumps/ Disallow: /maintenance/ Disallow: /temp/ Disallow: /cache/ Disallow: /systemlogs/ Allow: /fr/ Allow: /en/