# robots.txt for http://www.crucial.com User-agent: Morning Paper 1.0 (robots.txt compliant!) Disallow: / # User-agent: ia_archiver # Disallow: / User-agent: * Disallow: /images/ Disallow: /bin/ Disallow: /cache/ Disallow: /sandbox/ Disallow: /tools/ Disallow: /webservices/ Disallow: /websitedown/ Disallow: /w3c/ Disallow: /ajax/ Disallow: /common/ Disallow: /LexarEnterprise/ Disallow: /Master/ Disallow: /store/checkout/ Disallow: /eu/mysite/ Disallow: /uk/mysite/ Disallow: /us/mysite/ Disallow: /store/listmodule/