Google updates crawlers and user-triggered fetchers documentation

Posted on December 19, 2024

With that, Google expanded how Google products are affected by each crawler and gave robots.txt examples for each crawler.