User-agent: * Disallow: /*.asp$ Disallow: /public/under-maintenance.html Disallow: /public/service-unavailable.html Disallow: /public/401.html Disallow: /public/404.html Disallow: /public/500.html Disallow: /public/offer