Optimixed Search News

Cloudflare's Disallow AI Training feature allows sites to block AI scrapers without excluding Googlebot

65
News

TLDR

Cloudflare introduced a Disallow AI Training option that blocks AI crawlers from training data while preserving access for Googlebot and other search engines.

Optimixed Overview

Optimixed's original report on this story.

Cloudflare's new Disallow AI Training feature keeps mixed-use crawlers available for search indexing while allowing sites to opt out of AI model training. Google and Apple currently honor training opt-outs through this mechanism; Bing support for robots.txt directives in this area is still pending.

Topics:SEO News & Algorithm UpdatesAI & Machine Learning in SEO
Was this useful?
Share

Related articles

35
News

Google Launches Video Series on Tricky Technical SEO Questions

Google launched a Search Central video series answering unconventional technical SEO questions, beginning with topics like emoji, non-English characters, and URL keywords.

Technical SEOSEO News & Algorithm Updates
55
News

Google Adds Soft Navigation Metrics to CrUX Report

Google added experimental soft navigation metrics to the Chrome UX Report to track in-app page transitions, with Search Console support coming next.

Technical SEOSEO News & Algorithm UpdatesTools & Platforms
52
News

Google's Site Migration Processing Can Take 4+ Months

Google's John Mueller clarified that site migrations can take four months or longer to process fully, and offered guidance on what to prioritize during the transition.

Technical SEOSEO News & Algorithm Updates
35
News

HTML Sitemaps Still Have SEO Value, Google Explains

Google clarified that HTML sitemaps can still serve a purpose for SEO despite XML sitemaps becoming the standard.

Technical SEOSEO News & Algorithm Updates