Hy all.
I was wondering, if there is any way, to check, when Google spidered a site X. (X is not my site), or how often Google is Google spidering this site X?

I have an idea for start...I dont know if it is ok.. but in Google search often appears url: in cache. And you can see, when the cache was created. Well.. if you have to consecutive times from Google cache, it is simple to calculate how often (aproximately) Google is visiting this page?

Any ideas?

Recommended Answers

All 12 Replies

Hi
Spider activity is usually related to the amount of links to your site. More links would usually equal a fresher cache of your home page.

I am not sure... But I think you can keep track of that in Google Webmaster tools. I know that Google updates it's Ranking about every 3 months. So, the next one should be by the end of March if I am not wrong.

It depends how often you often the content of your site. The more often you update the site, the more the spider will crawl.

it totally depends on PR of a site & how early content is added or changes. google crawl static site once in a month while some news site as CNN crawled every minutes

Well..I know that it depends on many circumstances, but is there any way to check? For a site that is not mine?

Thanks

All about related to content and site map so update it regular.

Yes, depends on ammount of links (PR) and frequency of updates.

I agree with that of the ''Frequency of quality Content''. My site got an improved PR within the first 3 months of hosting the site, and its gradually in the top ten of Google.

Post frequently, and be conscious of the title keywords too.

Does adding an RSS feed help (from another site) so it looks like you are adding content all the time?

Hi, paptamas

Generally speaking, Google bots would frequently crawl those sites with higher PR(above 4) and more frequent updates. You could know if Google is crawling your sites or not via the admin area of your CMS usually.

all the best,

More links to your site would usually equal a fresher cache of your home page.

Be a part of the DaniWeb community

We're a friendly, industry-focused community of developers, IT pros, digital marketers, and technology enthusiasts meeting, networking, learning, and sharing knowledge.