Google lists Googlebot file limits for crawling

02/04/26
Source: Search Engine Land by Barry Schwartz. Read the original article

Google updated two of its help documents to clarify how much Googlebot can crawl.