User-agent: * Allow: / Disallow: /es/search # Sitemap files Sitemap: /es/sitemapblog.xml Sitemap: /en/sitemapblog.xml Sitemap: /fr/sitemapblog.xml # Crawl delay for heavy bots User-agent: AhrefsBot Crawl-delay: 10 User-agent: SemrushBot Crawl-delay: 10 User-agent: MJ12bot Crawl-delay: 10 User-agent: Baiduspider Crawl-delay: 10