Google Publishes Documentation on Allowing Google-Agent Access to Websites
TLDR
Google has released a new Search help document explaining how website owners can allow Google-Agent to access their sites.
Optimixed Overview
Optimixed's original report on this story.
Google has published a new Search help document on how to allow Google-Agent access to your website. In the document, Google describes Google-Agent as the name used by agents hosted on Google infrastructure to navigate the web and perform actions upon user request, as SERoundtable reported.
How we got here: earlier coverage
- Oct 22Google CEO Says AI Agents Will Access Websites Through Pages and APIs
Google's CEO indicated AI agents will access websites both through traditional page browsing and via APIs, with protocols offering better scalability but longer development cycles.
- Oct 21Google clarifies data-sharing controls in updated Search help documentation
Google updated its help docs to explicitly list existing data-sharing controls including robots meta tags, data-nosnippet, X-Robots-Tag, and Search generative AI controls.
- Oct 6Google Search Crawl Rate Doc Adds Retry-After HTTP & More
Google Search Crawl Rate Doc Adds Retry-After HTTP & More
- Oct 5Google Is Fighting SERP Tracking. AI Agents May Make It Worse via @sejournal, @martinibuster
Google Is Fighting SERP Tracking. AI Agents May Make It Worse via @sejournal, @martinibuster