User-agent: * Disallow: Disallow: /admin.asp$ Disallow: /_archive/*$ Disallow: /archive/*$ Disallow: /test/*$