# Website Sitemap Sitemap: https://www.x-com.nl/sitemap.xml User-agent: * Disallow: User-agent: LeGuideImgServer Disallow: / User-Agent: Baiduspider Disallow: / User-Agent: CazoodleBot Disallow: / User-Agent: Fasterfox Disallow: / User-Agent: Ingrid Crawl-delay: 20 User-Agent: Jyxobot Disallow: / User-Agent: MJ12bot Disallow: / User-Agent: ShopWiki Disallow: / User-agent: linkdexbot Disallow: / User-agent: Gigabot Disallow: / User-agent: CatchBot Disallow: / User-agent: ccbot Disallow: / User-agent: rogerbot Crawl-delay: 5 User-agent: Screaming Frog SEO Spider Crawl-delay: 5