# Language switcher /taal/?redirect=... is a 302 + noindex; keep all crawlers out. # Disallow is repeated in every group: a crawler only obeys its most specific group. User-agent: * Disallow: /taal/ Allow: / Sitemap: https://www.theory7.net/sitemap.xml # English (/en/) content sitemap Sitemap: https://www.theory7.net/en/sitemap.xml # English blog & landing pages (Laravel) Sitemap: https://www.theory7.net/en/blog-sitemap.xml Sitemap: https://www.theory7.net/en/pages-sitemap.xml # German (/de/) content sitemap Sitemap: https://www.theory7.net/de/sitemap.xml # German blog & landing pages (Laravel) Sitemap: https://www.theory7.net/de/blog-sitemap.xml Sitemap: https://www.theory7.net/de/pages-sitemap.xml # Blog & Pages sitemaps Sitemap: https://www.theory7.net/blog-sitemap.xml Sitemap: https://www.theory7.net/pages-sitemap.xml # SEO Tools sitemap Sitemap: https://www.theory7.net/tools-sitemap.xml # Support/Knowledge Base sitemaps Sitemap: https://www.theory7.net/sitemap-support.xml Sitemap: https://www.theory7.net/sitemap-support-nl.xml Sitemap: https://www.theory7.net/sitemap-support-en-us.xml Sitemap: https://www.theory7.net/sitemap-support-de.xml Sitemap: https://www.theory7.net/sitemap-support-fr.xml # AI Crawlers — expliciet toestaan User-agent: GPTBot Disallow: /taal/ Allow: / User-agent: ChatGPT-User Disallow: /taal/ Allow: / User-agent: Claude-Web Disallow: /taal/ Allow: / User-agent: Anthropic-AI Disallow: /taal/ Allow: / User-agent: Google-Extended Disallow: /taal/ Allow: / User-agent: CCBot Disallow: /taal/ Allow: / User-agent: PerplexityBot Disallow: /taal/ Allow: /