User-agent: * Disallow: /js Disallow: /files Disallow: /Scripts Disallow: /images Disallow: /cgi-binx Allow: / Disallow: /awstats Disallow: /swf