943,594 Members | Top Members by Rank

Ad:
Sep 14th, 2007
0

Google crawler frequency.

Expand Post »
Hi

On my website i have almost around 25000 webpages(static).
when i go to google and enter
inurl:mysite.com
it shows only 1000 pages. these are the pages crawled by google.
Here i notice that some of my webpages have never been crawled or never been indexed.
Also some of my webpages haven't been crawled since july 10, while some other pages have been crawled very recently(9 sept).

BTW it is PR 4 website.

can i do anything so that google crawls some more pages.
Last edited by devesh9392; Sep 14th, 2007 at 1:00 pm.
Similar Threads
Reputation Points: 9
Solved Threads: 0
Light Poster
devesh9392 is offline Offline
34 posts
since Apr 2007
Sep 14th, 2007
0

Re: Google crawler frequency.

The largest factor in determining how many are indexed are the PR of the individual pages. Not the PR of the homepage. You'll need to make sure you get deep links to your site as well as spreading the PR from your homepage around as well.

Also make sure all of your pages can be reached by the search engines. Just because you can get to them in your browser doesn't mean their crawlers can get to them.
Moderator
Reputation Points: 161
Solved Threads: 38
He's No Good To Me Dead
stymiee is offline Offline
1,422 posts
since May 2006
Sep 14th, 2007
0

Re: Google crawler frequency.

Click to Expand / Collapse  Quote originally posted by stymiee ...
You'll need to make sure you get deep links to your site as well as spreading the PR from your homepage around as well.

Can you please tell me what do you mean by "deep links" ?
and how can i sprear pr from my home page ? (by linking all important pages to home page ? )

thnx.
Reputation Points: 9
Solved Threads: 0
Light Poster
devesh9392 is offline Offline
34 posts
since Apr 2007
Sep 14th, 2007
0

Re: Google crawler frequency.

Is the correct META tags present?
Moderator
Featured Poster
Reputation Points: 1764
Solved Threads: 574
Moderator
jbennet is offline Offline
16,485 posts
since Apr 2005
Sep 15th, 2007
0

Re: Google crawler frequency.

Click to Expand / Collapse  Quote originally posted by devesh9392 ...
Can you please tell me what do you mean by "deep links" ?
A deep link is a link to an inner page in your site (e.g. not your home page).

Click to Expand / Collapse  Quote originally posted by devesh9392 ...
and how can i sprear pr from my home page ? (by linking all important pages to home page ? )
You can spread PR by linking to inner pages. Cross linking your pages is an important part of on site SEO.

Click to Expand / Collapse  Quote originally posted by jbennet ...
Is the correct META tags present?
Meta tags won't affect crawl rate. The most you can do is stop a page from being crawled and/or indexed.
Moderator
Reputation Points: 161
Solved Threads: 38
He's No Good To Me Dead
stymiee is offline Offline
1,422 posts
since May 2006
Sep 20th, 2007
0

Re: Google crawler frequency.

Click to Expand / Collapse  Quote originally posted by stymiee ...
Meta tags won't affect crawl rate. The most you can do is stop a page from being crawled and/or indexed.
do you mean if i have same meta tag keywords and description for all my pages,
it would be considered as spamming and google won't crawl any further ?

Thanks...
Reputation Points: 9
Solved Threads: 0
Light Poster
devesh9392 is offline Offline
34 posts
since Apr 2007
Sep 20th, 2007
0

Re: Google crawler frequency.

Click to Expand / Collapse  Quote originally posted by stymiee ...
You can spread PR by linking to inner pages. Cross linking your pages is an important part of on site SEO.

does it mean i should have more than one link to reach any page in my website ?
Reputation Points: 9
Solved Threads: 0
Light Poster
devesh9392 is offline Offline
34 posts
since Apr 2007
Sep 21st, 2007
0

Re: Google crawler frequency.

Click to Expand / Collapse  Quote originally posted by devesh9392 ...
do you mean if i have same meta tag keywords and description for all my pages,
it would be considered as spamming and google won't crawl any further ?Thanks...
No, it means that the meta robots tag should only be used to block content. It doesn't help it be crawled or indexed.

Click to Expand / Collapse  Quote originally posted by devesh9392 ...
does it mean i should have more than one link to reach any page in my website ?
As long as it makes sense to have those links, then yes.
Moderator
Reputation Points: 161
Solved Threads: 38
He's No Good To Me Dead
stymiee is offline Offline
1,422 posts
since May 2006

This thread is more than three months old

No one has posted to this discussion for at least three months. Please let old threads die and do not reply to them unless you feel you have something new and valuable to contribute that absolutely must be added to make the discussion complete. Otherwise, please start a new thread in this forum instead.
Message:
Previous Thread in Search Engine Optimization Forum Timeline: How to check fake PR.....?
Next Thread in Search Engine Optimization Forum Timeline: Sitemap questions





About Us | Contact Us | Advertise | Acceptable Use Policy
Forum Index | Build Custom RSS Feed


Follow us on Twitter


© 2011 DaniWeb® LLC