Thursday, October 15, 2009

Googlebot Crawl Rate


I took a look at how Google's spider was reading my site today. The search engine spider used by Google is called Googlebot. So far this month Googlebot has used up 43MB of bandwidth [as seen by AWSTATS]. Last month Googlebot used 64 MegaBytes of bandwidth via 7327 hits to the server. The attached graphic shows a high of 1,184 hits with an average of 557 hits and a low of 198 hits per day. The website update rate is;
Oct.: 202 page updates
Sep.: 336 pages updates
Aug.; 253 page updates
July; 165 page updates
So it would appear that Google is just barely picking up on the pages that get updated, if only because Googlebot will sometimes reread the same page more than once.

Related Posts;
Googlebot Crawl Stats; Jan 11 2009.
Spider Crawl Stats; Oct 24 2008.
Googlebot Crawl Stats; July 4 2008.
Robot Visits; Sep 14 2007.
Google Crawl Rate; April 14 2007.
Each one of these older posts provides a graphic of the crawl rate.

No comments:

Post a Comment