943,513 Members | Top Members by Rank

Ad:
You are currently viewing page 1 of this multi-page discussion thread
Aug 22nd, 2003
0

I want google to index forum threads

Expand Post »
We all know that google doesn't like to index dynamic pages. However, I just checked up on the latest daniweb.com pages indexed by google, and guess what I found?

Google has indexed just about every member's profile page, but not the forum threads! The member profile pages contain less original content, and contain the same number of variables in the URL (e.g. member.php?u=1 as opposed to showthread.php?p=100) Why is this, and how could I get google to index the important pages instead of the meaningless ones?
Similar Threads
Administrator
Staff Writer
Reputation Points: 1422
Solved Threads: 162
The Queen of DaniWeb
cscgal is online now Online
13,645 posts
since Feb 2002
Aug 23rd, 2003
0

Re: I want google to index forum threads

that is very odd. i'm sure you know your site better than anyone so think about a site map in your mind. think how google could have gotten there without seeing/indexing these pages. that's the only advice i can give.
Reputation Points: 11
Solved Threads: 3
Junior Poster in Training
evilmonkey29 is offline Offline
71 posts
since Aug 2002
Sep 12th, 2003
0
Re: I want google to index forum threads
Google indexes my forums threads, perhaps you have a robots.txt that is filtering out showthread.php.

Im running phpBB, but I dont think that should matter. Definatly check to see if there is a robots.txt file anywhere in your public_html folder. If you find one, check to see if showthread.php is located anywhere.
Reputation Points: 10
Solved Threads: 0
Newbie Poster
jibtronic is offline Offline
13 posts
since Sep 2003
Sep 12th, 2003
0

Re: I want google to index forum threads

Nope, no robots.txt. I'm putting some more effort into this since a few weeks ago when I first posted, and it's getting a bit better. A number of threads are indexed now.

Jibtronic, have you tried using the phpBB search-engine friendly hack? It's awfully good. You can find it via phphacks.com and it's the one that uses mod rewrite and an .htaccess file.

It lets you convert all viewtopic.php?t=100 threads into URLs like t100.html because we all know how much google likes static URLs better than dynamic URLs.

I actually wrote my own version of this mod for my own use awhile back (when this forum ran off of phpBB). But I never released it. Guess someone else finally did
Administrator
Staff Writer
Reputation Points: 1422
Solved Threads: 162
The Queen of DaniWeb
cscgal is online now Online
13,645 posts
since Feb 2002
Mar 2nd, 2004
0

Re: I want google to index forum threads

Google will now log variables but it has some blocked like ?id=(n) so I guess that ?p= is one of the ones that are blocked
Reputation Points: 10
Solved Threads: 0
Junior Poster in Training
Ragnarok is offline Offline
94 posts
since Mar 2004
Mar 16th, 2004
0

Re: I want google to index forum threads

Google doesn't necessarily have certain variables such as ?id= blocked. What it is is that Google doesn't want to index URLs containing a session ID (that long value like 8482349809934 ...) ... if you don't know what I'm talking about, check out any page on Amazon.com

In any case, Google doesn't block variables such as ?id= or ?sid= but rather any variable that has a value that resembles a session ID.

Regardless, I found out my problem. I was doing a google serch for site:www.daniweb.com daniweb to return all URLs in the daniweb.com domain that contain the word "daniweb" - basically to estimate how many of my pages are indexed with Google. Apparently my member profile pages just had the word "daniweb" more prominently than my other pages, which wouldn't display until I clicked on the "show all pages" link at the end of the Google search results.

Regardless, guests have now been prohibited from viewing member pages, so Google won't see this anymore anyways site:www.daniweb.com daniweb
Last edited by cscgal; Mar 16th, 2004 at 2:05 pm.
Administrator
Staff Writer
Reputation Points: 1422
Solved Threads: 162
The Queen of DaniWeb
cscgal is online now Online
13,645 posts
since Feb 2002
Mar 20th, 2004
0

Re: I want google to index forum threads

Do this:
allinurl:
instead of
site:
Reputation Points: 10
Solved Threads: 0
Newbie Poster
carolblake1973 is offline Offline
11 posts
since Mar 2004
Mar 22nd, 2004
0

Re: I want google to index forum threads

vBulletin3 is exceptionally good for SEO purposes. Good choice, cscgal.
Reputation Points: 10
Solved Threads: 1
Light Poster
I, Brian is offline Offline
47 posts
since Mar 2004
Apr 21st, 2004
0

Re: I want google to index forum threads

Hi Brian.

Seeing you here suggests something about the value of the forum links.

By the way I can see GPR 3 on this very thread, so it is indexed.
Reputation Points: 10
Solved Threads: 0
Newbie Poster
scisoft is offline Offline
7 posts
since Apr 2004
May 10th, 2004
0

Re: I want google to index forum threads

I've found that time is the best thing for indexing forums. As long as a hack is installed. Your pages will all get indexed eventually.
Reputation Points: 8
Solved Threads: 0
Newbie Poster
nacho45 is offline Offline
15 posts
since May 2004

This thread is more than three months old

No one has posted to this discussion for at least three months. Please let old threads die and do not reply to them unless you feel you have something new and valuable to contribute that absolutely must be added to make the discussion complete. Otherwise, please start a new thread in this forum instead.
Message:
Previous Thread in Search Engine Optimization Forum Timeline: Tags on Posts
Next Thread in Search Engine Optimization Forum Timeline: Local search engine help





About Us | Contact Us | Advertise | Acceptable Use Policy
Forum Index | Build Custom RSS Feed


Follow us on Twitter


© 2011 DaniWeb® LLC