User Name Password Register
DaniWeb IT Discussion Community
All
What is DaniWeb IT Discussion Community?
You're currently browsing the Search Engine Optimization section within the Site Management category of DaniWeb, a massive community of 423,607 software developers, web developers, Internet marketers, and tech gurus who are all enthusiastic about making contacts, networking, and learning from each other. In fact, there are 3,231 IT professionals currently interacting right now! Registration is free, only takes a minute and lets you enjoy all of the interactive features of the site.
Please support our Search Engine Optimization advertiser: Affiliate Marketing
Views: 7107 | Replies: 10
Reply
Join Date: Feb 2002
Location: Lawn Guylen, NY
Posts: 10,900
Reputation: cscgal is just really nice cscgal is just really nice cscgal is just really nice cscgal is just really nice cscgal is just really nice 
Rep Power: 32
Solved Threads: 118
Admin
Staff Writer
cscgal's Avatar
cscgal cscgal is offline Offline
The Queen of DaniWeb

I want google to index forum threads

  #1  
Aug 22nd, 2003
We all know that google doesn't like to index dynamic pages. However, I just checked up on the latest daniweb.com pages indexed by google, and guess what I found?

Google has indexed just about every member's profile page, but not the forum threads! The member profile pages contain less original content, and contain the same number of variables in the URL (e.g. member.php?u=1 as opposed to showthread.php?p=100) Why is this, and how could I get google to index the important pages instead of the meaningless ones?
AddThis Social Bookmark Button
Reply With Quote  
Join Date: Jul 2002
Posts: 71
Reputation: evilmonkey29 is an unknown quantity at this point 
Rep Power: 7
Solved Threads: 3
evilmonkey29 evilmonkey29 is offline Offline
Junior Poster in Training

Re: I want google to index forum threads

  #2  
Aug 22nd, 2003
that is very odd. i'm sure you know your site better than anyone so think about a site map in your mind. think how google could have gotten there without seeing/indexing these pages. that's the only advice i can give.
Reply With Quote  
Join Date: Sep 2003
Location: Canada
Posts: 13
Reputation: jibtronic is an unknown quantity at this point 
Rep Power: 6
Solved Threads: 0
jibtronic jibtronic is offline Offline
Newbie Poster
  #3  
Sep 12th, 2003
Google indexes my forums threads, perhaps you have a robots.txt that is filtering out showthread.php.

Im running phpBB, but I dont think that should matter. Definatly check to see if there is a robots.txt file anywhere in your public_html folder. If you find one, check to see if showthread.php is located anywhere.
Reply With Quote  
Join Date: Feb 2002
Location: Lawn Guylen, NY
Posts: 10,900
Reputation: cscgal is just really nice cscgal is just really nice cscgal is just really nice cscgal is just really nice cscgal is just really nice 
Rep Power: 32
Solved Threads: 118
Admin
Staff Writer
cscgal's Avatar
cscgal cscgal is offline Offline
The Queen of DaniWeb

Re: I want google to index forum threads

  #4  
Sep 12th, 2003
Nope, no robots.txt. I'm putting some more effort into this since a few weeks ago when I first posted, and it's getting a bit better. A number of threads are indexed now.

Jibtronic, have you tried using the phpBB search-engine friendly hack? It's awfully good. You can find it via phphacks.com and it's the one that uses mod rewrite and an .htaccess file.

It lets you convert all viewtopic.php?t=100 threads into URLs like t100.html because we all know how much google likes static URLs better than dynamic URLs.

I actually wrote my own version of this mod for my own use awhile back (when this forum ran off of phpBB). But I never released it. Guess someone else finally did
Reply With Quote  
Join Date: Mar 2004
Location: In a house
Posts: 94
Reputation: Ragnarok is an unknown quantity at this point 
Rep Power: 5
Solved Threads: 0
Ragnarok Ragnarok is offline Offline
Junior Poster in Training

Re: I want google to index forum threads

  #5  
Mar 2nd, 2004
Google will now log variables but it has some blocked like ?id=(n) so I guess that ?p= is one of the ones that are blocked
Reply With Quote  
Join Date: Feb 2002
Location: Lawn Guylen, NY
Posts: 10,900
Reputation: cscgal is just really nice cscgal is just really nice cscgal is just really nice cscgal is just really nice cscgal is just really nice 
Rep Power: 32
Solved Threads: 118
Admin
Staff Writer
cscgal's Avatar
cscgal cscgal is offline Offline
The Queen of DaniWeb

Re: I want google to index forum threads

  #6  
Mar 16th, 2004
Google doesn't necessarily have certain variables such as ?id= blocked. What it is is that Google doesn't want to index URLs containing a session ID (that long value like 8482349809934 ...) ... if you don't know what I'm talking about, check out any page on Amazon.com

In any case, Google doesn't block variables such as ?id= or ?sid= but rather any variable that has a value that resembles a session ID.

Regardless, I found out my problem. I was doing a google serch for site:www.daniweb.com daniweb to return all URLs in the daniweb.com domain that contain the word "daniweb" - basically to estimate how many of my pages are indexed with Google. Apparently my member profile pages just had the word "daniweb" more prominently than my other pages, which wouldn't display until I clicked on the "show all pages" link at the end of the Google search results.

Regardless, guests have now been prohibited from viewing member pages, so Google won't see this anymore anyways site:www.daniweb.com daniweb
Last edited by cscgal : Mar 16th, 2004 at 1:05 pm.
Reply With Quote  
Join Date: Mar 2004
Posts: 11
Reputation: carolblake1973 is an unknown quantity at this point 
Rep Power: 5
Solved Threads: 0
carolblake1973 carolblake1973 is offline Offline
Newbie Poster

Re: I want google to index forum threads

  #7  
Mar 20th, 2004
Do this:
allinurl:
instead of
site:
Reply With Quote  
Join Date: Mar 2004
Posts: 47
Reputation: I, Brian is an unknown quantity at this point 
Rep Power: 5
Solved Threads: 1
I, Brian's Avatar
I, Brian I, Brian is offline Offline
Light Poster
Join Date: Apr 2004
Posts: 7
Reputation: scisoft is an unknown quantity at this point 
Rep Power: 0
Solved Threads: 0
scisoft scisoft is offline Offline
Newbie Poster

Re: I want google to index forum threads

  #9  
Apr 21st, 2004
Hi Brian.

Seeing you here suggests something about the value of the forum links.

By the way I can see GPR 3 on this very thread, so it is indexed.
SEO
Reply With Quote  
Join Date: May 2004
Location: Moscow
Posts: 15
Reputation: nacho45 is an unknown quantity at this point 
Rep Power: 0
Solved Threads: 0
nacho45 nacho45 is offline Offline
Newbie Poster

Re: I want google to index forum threads

  #10  
May 10th, 2004
I've found that time is the best thing for indexing forums. As long as a hack is installed. Your pages will all get indexed eventually.
Reply With Quote  
Reply

Only community members can participate in forum threads. You must register or log in to contribute.

DaniWeb Search Engine Optimization Marketplace
Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)

 

Thread Tools Display Modes

Other Threads in the Search Engine Optimization Forum

All times are GMT -4. The time now is 8:35 pm.
Forum system based on vBulletin Copyright ©2000 - 2008, Jelsoft Enterprises Ltd.
©2003 - 2008 DaniWeb® LLC