•
•
•
•
What is DaniWeb IT Discussion Community?
You're currently browsing the Search Engine Optimization section within the Site Management category of DaniWeb, a massive community of 402,006 software developers, web developers, Internet marketers, and tech gurus who are all enthusiastic about making contacts, networking, and learning from each other. In fact, there are 2,387 IT professionals currently interacting right now! Registration is free, only takes a minute and lets you enjoy all of the interactive features of the site.
Please support our Search Engine Optimization advertiser: Affiliate Marketing
Views: 6413 | Replies: 22
 |
•
•
Join Date: Feb 2002
Location: Lawn Guylen, NY
Posts: 10,889
Reputation: 
Rep Power: 32
Solved Threads: 109
I would like to maximize the PR on my forum display and thread display pages, without sacrificing PR to less important pages. (for instance, the newthread.php page)
Now suppose there are 5 links on page A. Normally, page A's PR will be spread out to the 5 pages. But now if I added a robots.txt file which blocked indexing of two of the links. Would page A's PR now be spread less thin among 3 pages? Or would it be spread just as thin, but 2 of the pages would be entitled to a share of PR but just wouldn't use it.
I hope this sorta made a bit of sense.
Now suppose there are 5 links on page A. Normally, page A's PR will be spread out to the 5 pages. But now if I added a robots.txt file which blocked indexing of two of the links. Would page A's PR now be spread less thin among 3 pages? Or would it be spread just as thin, but 2 of the pages would be entitled to a share of PR but just wouldn't use it.
I hope this sorta made a bit of sense.
Dani the Computer Science Gal
Do you run a computer-related website? Feature it in our niche link directory!
Do you run a computer-related website? Feature it in our niche link directory!
•
•
Join Date: Jun 2004
Location: Phoenix Arizona
Posts: 115
Reputation:
Rep Power: 5
Solved Threads: 2
Junior Poster
I sent you a PM RE: this topic. 
Need a website designer? arizona web design : phoenix web design : MCP Media intelligent web design and web development solutions. MCP Media is owned and operated by Chris Hooley - who happens to be a real nerd... on purpose :-)
•
•
Join Date: Feb 2002
Location: Lawn Guylen, NY
Posts: 10,889
Reputation:
Rep Power: 32
Solved Threads: 109
And I started this thread RE: your PM
Dani the Computer Science Gal
Do you run a computer-related website? Feature it in our niche link directory!
Do you run a computer-related website? Feature it in our niche link directory!
•
•
Join Date: Jun 2004
Location: Phoenix Arizona
Posts: 115
Reputation:
Rep Power: 5
Solved Threads: 2
Junior Poster
Ok I am a knuckelhead. I guess this was a chicken before the egg thing. ;-)
Need a website designer? arizona web design : phoenix web design : MCP Media intelligent web design and web development solutions. MCP Media is owned and operated by Chris Hooley - who happens to be a real nerd... on purpose :-)
•
•
Join Date: Feb 2002
Location: Lawn Guylen, NY
Posts: 10,889
Reputation:
Rep Power: 32
Solved Threads: 109
The robots.txt file I'm currently using is in my root directory (not my forum root) and looks like this:
I am disallowing access to showthread.php and forumdisplay.php because I would rather Google only spider the .html mod_rewrite versions of the forums and threads, and therefore not get duplicate content. Was this done correctly? Am I excluding the correct things?
User-agent: googlebot Disallow: /techtalkforums/announcement.php Disallow: /techtalkforums/faq.php Disallow: /techtalkforums/forumdisplay.php Disallow: /techtalkforums/login.php Disallow: /techtalkforums/member.php Disallow: /techtalkforums/newreply.php Disallow: /techtalkforums/newthread.php Disallow: /techtalkforums/online.php Disallow: /techtalkforums/printthread.php Disallow: /techtalkforums/search.php Disallow: /techtalkforums/showthread.php
I am disallowing access to showthread.php and forumdisplay.php because I would rather Google only spider the .html mod_rewrite versions of the forums and threads, and therefore not get duplicate content. Was this done correctly? Am I excluding the correct things?
Dani the Computer Science Gal
Do you run a computer-related website? Feature it in our niche link directory!
Do you run a computer-related website? Feature it in our niche link directory!
•
•
Join Date: Jun 2004
Location: Phoenix Arizona
Posts: 115
Reputation:
Rep Power: 5
Solved Threads: 2
Junior Poster
without going too far in, it looks decent to me. I am not sure however, if the robots.txt blocks the weakening link pop from all the links. The less links on a page the more potent the links are. A page with tons of links is spreading the pop thin. That would be a good question to ask SEO-Guy.
Need a website designer? arizona web design : phoenix web design : MCP Media intelligent web design and web development solutions. MCP Media is owned and operated by Chris Hooley - who happens to be a real nerd... on purpose :-)
•
•
Join Date: Feb 2002
Location: Lawn Guylen, NY
Posts: 10,889
Reputation:
Rep Power: 32
Solved Threads: 109
It would be very nice if the robots.txt would block the weakening spread of PR. However, even if it doesn't do this, it would still be valuable because it would eliminate spidering duplicate content (i.e. showthread.php?t=10 and thread10.html)
Dani the Computer Science Gal
Do you run a computer-related website? Feature it in our niche link directory!
Do you run a computer-related website? Feature it in our niche link directory!
•
•
Join Date: Jun 2004
Location: Phoenix Arizona
Posts: 115
Reputation:
Rep Power: 5
Solved Threads: 2
Junior Poster
•
•
•
•
Originally Posted by cscgal
It would be very nice if the robots.txt would block the weakening spread of PR. However, even if it doesn't do this, it would still be valuable because it would eliminate spidering duplicate content (i.e. showthread.php?t=10 and thread10.html)
Need a website designer? arizona web design : phoenix web design : MCP Media intelligent web design and web development solutions. MCP Media is owned and operated by Chris Hooley - who happens to be a real nerd... on purpose :-)
•
•
Join Date: Feb 2002
Location: Lawn Guylen, NY
Posts: 10,889
Reputation:
Rep Power: 32
Solved Threads: 109
Google frowns upon multiple pages with the same content. For example, if two different URLs have the exact same content on them, google considers it spamming their search engine. This forum uses Apache's mod_rewrite to rewrite URLs to have a .html extension for search engine purposes. Therefore, the webpage showthread.php?t=100 is the exact same thing as thread100.html - if google spiders see this duplicate contact, they will think that daniweb.com is trying to inflate its page count in google by having multiple URLs with the same content. However, by using robots.txt to block google from spidering the showthread.php pages, google only spiders the pages ending in .html - and therefore doesn't penalize us for duplicate content.
Dani the Computer Science Gal
Do you run a computer-related website? Feature it in our niche link directory!
Do you run a computer-related website? Feature it in our niche link directory!
|
•
•
•
•
•
•
•
•
DaniWeb Search Engine Optimization Marketplace
•
•
•
•
Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
Linear Mode
•
•
•
•
adsense adult advertising blogging bomb book business coding community copyright development engine environment external forum gasoline google hybrid internal internal combustion engine legal links malware marketing mcafee microsoft monetization msn news pagerank patent privacy publishing revenue search security seo sex silverlight software spyware support technical tools universe video web webmaster wiki yahoo
- Previous Thread: Help.
- Next Thread: Search Engines into LSI
