Google crawler frequency.

Please support our Search Engine Optimization advertiser: Get a Free SEO Analysis!
Reply

Join Date: Apr 2007
Posts: 34
Reputation: devesh9392 is an unknown quantity at this point 
Solved Threads: 0
devesh9392 devesh9392 is offline Offline
Light Poster

Google crawler frequency.

 
0
  #1
Sep 14th, 2007
Hi

On my website i have almost around 25000 webpages(static).
when i go to google and enter
inurl:mysite.com
it shows only 1000 pages. these are the pages crawled by google.
Here i notice that some of my webpages have never been crawled or never been indexed.
Also some of my webpages haven't been crawled since july 10, while some other pages have been crawled very recently(9 sept).

BTW it is PR 4 website.

can i do anything so that google crawls some more pages.
Last edited by devesh9392; Sep 14th, 2007 at 1:00 pm.
Reply With Quote Quick reply to this message  
Join Date: May 2006
Posts: 1,422
Reputation: stymiee is on a distinguished road 
Solved Threads: 35
Moderator
stymiee's Avatar
stymiee stymiee is offline Offline
He's No Good To Me Dead

Re: Google crawler frequency.

 
0
  #2
Sep 14th, 2007
The largest factor in determining how many are indexed are the PR of the individual pages. Not the PR of the homepage. You'll need to make sure you get deep links to your site as well as spreading the PR from your homepage around as well.

Also make sure all of your pages can be reached by the search engines. Just because you can get to them in your browser doesn't mean their crawlers can get to them.
John Conde
Brainyminds | Merchant Account Services | I Love Code
IT'S HERE: Merchant Accounts 101 Everything you need to know about merchant accounts!
Reply With Quote Quick reply to this message  
Join Date: Apr 2007
Posts: 34
Reputation: devesh9392 is an unknown quantity at this point 
Solved Threads: 0
devesh9392 devesh9392 is offline Offline
Light Poster

Re: Google crawler frequency.

 
0
  #3
Sep 14th, 2007
Originally Posted by stymiee View Post
You'll need to make sure you get deep links to your site as well as spreading the PR from your homepage around as well.

Can you please tell me what do you mean by "deep links" ?
and how can i sprear pr from my home page ? (by linking all important pages to home page ? )

thnx.
Reply With Quote Quick reply to this message  
Join Date: Apr 2005
Posts: 16,146
Reputation: jbennet is a name known to all jbennet is a name known to all jbennet is a name known to all jbennet is a name known to all jbennet is a name known to all jbennet is a name known to all 
Solved Threads: 530
Moderator
Featured Poster
jbennet's Avatar
jbennet jbennet is offline Offline
Moderator

Re: Google crawler frequency.

 
0
  #4
Sep 14th, 2007
Is the correct META tags present?
If i am helpful, please give me reputation points.
Reply With Quote Quick reply to this message  
Join Date: May 2006
Posts: 1,422
Reputation: stymiee is on a distinguished road 
Solved Threads: 35
Moderator
stymiee's Avatar
stymiee stymiee is offline Offline
He's No Good To Me Dead

Re: Google crawler frequency.

 
0
  #5
Sep 15th, 2007
Originally Posted by devesh9392 View Post
Can you please tell me what do you mean by "deep links" ?
A deep link is a link to an inner page in your site (e.g. not your home page).

Originally Posted by devesh9392 View Post
and how can i sprear pr from my home page ? (by linking all important pages to home page ? )
You can spread PR by linking to inner pages. Cross linking your pages is an important part of on site SEO.

Originally Posted by jbennet View Post
Is the correct META tags present?
Meta tags won't affect crawl rate. The most you can do is stop a page from being crawled and/or indexed.
John Conde
Brainyminds | Merchant Account Services | I Love Code
IT'S HERE: Merchant Accounts 101 Everything you need to know about merchant accounts!
Reply With Quote Quick reply to this message  
Join Date: Apr 2007
Posts: 34
Reputation: devesh9392 is an unknown quantity at this point 
Solved Threads: 0
devesh9392 devesh9392 is offline Offline
Light Poster

Re: Google crawler frequency.

 
0
  #6
Sep 20th, 2007
Originally Posted by stymiee View Post
Meta tags won't affect crawl rate. The most you can do is stop a page from being crawled and/or indexed.
do you mean if i have same meta tag keywords and description for all my pages,
it would be considered as spamming and google won't crawl any further ?

Thanks...
Reply With Quote Quick reply to this message  
Join Date: Apr 2007
Posts: 34
Reputation: devesh9392 is an unknown quantity at this point 
Solved Threads: 0
devesh9392 devesh9392 is offline Offline
Light Poster

Re: Google crawler frequency.

 
0
  #7
Sep 20th, 2007
Originally Posted by stymiee View Post
You can spread PR by linking to inner pages. Cross linking your pages is an important part of on site SEO.

does it mean i should have more than one link to reach any page in my website ?
Reply With Quote Quick reply to this message  
Join Date: May 2006
Posts: 1,422
Reputation: stymiee is on a distinguished road 
Solved Threads: 35
Moderator
stymiee's Avatar
stymiee stymiee is offline Offline
He's No Good To Me Dead

Re: Google crawler frequency.

 
0
  #8
Sep 21st, 2007
Originally Posted by devesh9392 View Post
do you mean if i have same meta tag keywords and description for all my pages,
it would be considered as spamming and google won't crawl any further ?Thanks...
No, it means that the meta robots tag should only be used to block content. It doesn't help it be crawled or indexed.

Originally Posted by devesh9392 View Post
does it mean i should have more than one link to reach any page in my website ?
As long as it makes sense to have those links, then yes.
John Conde
Brainyminds | Merchant Account Services | I Love Code
IT'S HERE: Merchant Accounts 101 Everything you need to know about merchant accounts!
Reply With Quote Quick reply to this message  
Reply

This thread is more than three months old.
Perhaps start a new thread instead?
Message:



Similar Threads
Other Threads in the Search Engine Optimization Forum
Thread Tools Search this Thread



About Us | Contact Us | Advertise | DaniWeb | Acceptable Use Policy | RSS Feed

©2003 - 2009 DaniWeb® LLC