# Crawl-delay is honoured by AhrefsBot and SemrushBot (Semrush caps it at 10s).
# GPTBot does not document or honour Crawl-delay - OpenAI's bot docs list no
# rate-limit directive - so slowing it needs Disallow or an edge rule instead.
# Note: a bot matching its own User-agent group ignores the * group entirely,
# so each block below repeats the rules it still needs.

User-agent: AhrefsBot
Allow: /
Disallow: /admin/
Disallow: /search
Crawl-delay: 10

User-agent: SemrushBot
Allow: /
Disallow: /admin/
Disallow: /search
Crawl-delay: 10

User-agent: *
Allow: /
Disallow: /admin/
Disallow: /search
Sitemap: https://opencompanies.uk/sitemap.xml