# All robots will spider the domain User-agent: * Disallow: # Disallow directories User-agent: * Disallow: /steyning/ Disallow: /shoreham/ Disallow: /cgi-bin/ Disallow: /Connections/ Disallow: /css/ Disallow: /images/ Disallow: /js/ Disallow: /SpryAssets/