User-agent: * Disallow: /WebResource.axd User-agent: Jooblebot # Jooblebot - scrapes job adverts and re-publishes on own site - details here https://jooble.org/jooble-bot User-agent: 8LEGS # 8legs.com - scrapes job adverts User-agent: PiplBot # Piplbot - search engine for finding people. https://pipl.com/bot/ User-agent: ExaBot # Exabot - unknown source - 403 forbidden at http://www.exabot.com/ Disallow: /