The next one bars all robots from the cgi-bin and archive directories: User-agent: * Disallow: /cgi-bin/ Disallow: /archive/ Disallow: /registration/ Disallow: /certificates/ Disallow: /bsts-old/ Disallow: /common/ Disallow: /eBUG/ Disallow: /fieldformat/ Disallow: /resourcecontent/ Disallow: /test-lab/ Disallow: /news-archives/ Disallow: /resource1/ Disallow: /resourcecontent/ Disallow: /LicWebService/ Disallow: /N2XWebService/ Disallow: /lwcatalog/csvs/