Member Avatar for olddocks

I am totally tired with Vbulletin/phpBB2 forums, so i decided to code a forum application on my own named "Balakrishnan Forum v1.0", with simple to use and good looking features. I applied my forum on these websites



http://www.apostilleinfo.com/forum/

But the problem i face is all my site pages are indexed by google/yahoo/msn, except all inner forum pages. I also removed any session ids that is generated automatically. Just could'nt figure out what the problem! Can anybody help me?

  • 3 Contributors
  • 5 Replies
  • 200 Views
  • 1 Week Discussion Span
  • Latest Post Latest Post by olddocks

Dani AI

Generated

Short summary and likely cause (based on the thread): when a site’s homepage and top-level pages are indexed but deeper forum pages aren’t, the culprit is usually crawl/access signals or URL duplication (session IDs, tracking parameters, faceted URLs) rather than a mysterious “Google blocker.” already found and removed session IDs — that is exactly the kind of low-value URL noise that reduces Google’s crawl efficiency and often prevents deep pages from being indexed. (developers.google.com)

Quick diagnostic checklist (do these in order):

  • Check robots.txt at site root to make sure you are NOT disallowing the forum paths and that a sitemap is referenced.
  • Confirm no unexpected <meta name="robots" content="noindex"> or X‑Robots‑Tag headers on thread pages.
  • Verify every thread/page is reachable with plain anchor <a href="..."> links (no form-only or JS-only access).
  • Submit an XML sitemap (only the URLs you want indexed) and use Search Console’s URL inspection to see crawl/index reasons.

Example robots + meta examples:

User-agent: *
Disallow:

Sitemap: https://example.com/sitemap.xml
<head>
  <meta name="robots" content="index, follow">
</head>

See Google’s guidance on robots/meta and sitemaps. (developers.google.com)

Fixes for parameter/session-id problems and duplicates:

  • Stop putting session ids into public URLs (you already did this; correct).
  • Where parameterized URLs still exist, use 301 redirects or a self-referencing rel="canonical" to the clean thread URL so Google sees one preferred URL. Example:
<link rel="canonical" href="https://forum.example.com/thread/1234/" />

Prefer redirects for permanent consolidation, then canonical as a secondary signal. Note: Google removed the old URL‑Parameters tool, so don’t rely on that — fix URLs and signals at source. (developers.google.com)

Pagination, internal linking and timing:

  • Make each paginated forum page stand on its own (unique title/meta, enough visible content). Google no longer treats rel="prev/next" as a guarantee, so don’t depend on it to force indexing; instead ensure good HTML linking and consider a “view-all” page if threads are short enough.
  • Build an HTML index (or topic index) and link to older pages from category pages so deep pages aren’t orphaned. After fixes, use Search Console’s URL Inspection / Request Indexing and watch the Coverage report; indexing can take days to weeks and Google may still skip low-value pages. (merj.com)

Practical next steps (short):

  1. Verify robots/meta and remove any blocks.
  2. Submit a clean XML sitemap and inspect a few thread URLs in Search Console.
  3. Canonicalize/301 redirect parameterized versions to clean URLs.
  4. Add internal links / HTML index so every thread is reachable.
    Do those and monitor Search Console coverage — that combination fixes ~95% of forum-indexing problems.

Recommended Answers

All 5 Replies

Member Avatar for iwonder

Use mod rewrite so that your pages are edited from php to html extension and then shown to user . It`s something you do with .htaccess file ( i`m not that good at php :o )

Member Avatar for olddocks

First i thank you iwonder for your reply.

yep! I already did that and my forums are fine tuned for htm pages using mod_rewrite. The mistake i did was earlier the session ids were automatically passed to htm pages. I quickly noticed that and quickly disabled those automatically generated session ids. To my surprise i found that just 2 days before google indexed my pages. I guess i have to wait about 2 months to get my sites completely indexed.

Let me know if you have other suggestions!

Member Avatar for jewboy

PageRank determines how deep Googlebot will crawl into your site. My advice would be to increase your PR with high quality inbound links.

Member Avatar for iwonder

I guess i have to wait about 2 months to get my sites completely indexed.

Well u can`t be sure , As promediacorp said inbound links do matter much . Suppose you have linked from 4 site`s it means Search Engines (SE) come`s to your site 4 time`s while crawling them . As far as I know Page rank depends on this .

Member Avatar for olddocks

I agree with your point, iwonder because from my experience i waited for about 2 months for one of my website and Google indexed 1600 pages at a go. I never promoted those sites at all.

I have no problem with indexing pages, but the main issue here is inner forum pages. There has been a lots of people saying that to get your inner pages indexed there should be...

1. Higher PR (of course depends of incoming links)
2. Allow plenty of time to search engines for google' LEVEL 2 crawl.

I think i agree with (2) though i cannot artifcially increase the PR of my pages. I am not that good in creating inbound links. Your advise in this, matter a lot for me.

Be a part of the DaniWeb community

We're a friendly, industry-focused community of developers, IT pros, digital marketers, and technology enthusiasts meeting, networking, learning, and sharing knowledge.