Member Avatar for posterreal

i request help on this topic: MSN, Google and Yahoo Directory currently ignore on of my clients website. None of the other search engines seems to have trouble.

The index.html is already in the three above. From there on, a cms is called through a frameset looking like this:

frame top= /cgi-bin/cms.pl?template=start
frame bottom= /cgi-bin/cms.pl?template=foot

These two pages are searchable as well (along with index.html) in Google, MSN and Yahoo, but none of the following pages.

So, i have a total of three pages , while some hundred are waiting since november 2002. The sites deeplinks look like this:

<a href = /cgi-bin/cms.pl?template=article //for categories
<a href = /cgi-bin/cms.pl?template=article&id=1234 //for articles

I used webmasterplan.de with their "this is how your site looks to engines" tool and found out that this tool requires "=" to be replaced by "%3d" (ascii opcode for equal signs). is this the same for google and company ?

  • 3 Contributors
  • 2 Replies
  • 124 Views
  • 2 Months Discussion Span
  • Latest Post Latest Post by MktgRob

Dani AI

Generated

For : this looks like a classic frames + dynamic-URL crawlability problem. When a site serves the CMS pages inside a frameset and the deep content is only reachable by query-string URLs, many crawlers will index the frameset (the wrapper) but not treat each framed page as a distinct, discoverable URL. That behavior explains why only a handful of pages appear in the index while hundreds of articles remain unseen. Frames are obsolete for modern sites and should be avoided for SEO (see the frameset guidance). (https://developer.mozilla.org/en-US/docs/Web/HTML/Element/frameset)

Quick diagnostic checklist

  • Confirm the article URLs return HTTP 200 (not 302/403/404) and are reachable without JavaScript or a frame wrapper.
  • Check robots.txt and meta robots for accidental blocks. Example minimal robots entry:
    
User-agent: *
Disallow:

Sitemap: 

- Use Search Console (URL Inspection) and Bing Webmaster Tools to "fetch/render" representative article URLs and to see how bots view the pages.  
- Inspect server logs for Googlebot/Bingbot requests to those article URLs.

Practical fixes, in priority order
- Eliminate the frameset or provide direct, crawlable URLs for every article. If the CMS supports friendly/permalink URLs or rewrite rules, enable them. If not, create an index page that links to each article with plain <a> links and submit a sitemap. Sitemaps are effective for telling crawlers about URL sets (protocol reference). (https://www.sitemaps.org/protocol.html)
- Add canonical tags on each article to a single preferred URL:

<link rel="canonical" href="" />


- If query parameters must remain, use the URL Parameters controls in Search Console to tell Google how parameters affect content. (https://support.google.com/webmasters/answer/6080550)

About the "%3d" (percent-encoding)
"=" is a reserved character in URIs and may be percent-encoded in some tools or contexts; RFC 3986 explains when encoding is required. In typical query strings "=" is the key/value separator and does not need encoding for crawlers to understand the URL. (https://datatracker.ietf.org/doc/html/rfc3986)

Mention for @MktgRob: knowing which CMS and whether it can produce friendly URLs or an exportable sitemap will pinpoint the fastest fix.

Recommended Answers

All 2 Replies

Member Avatar for makesaw

I dont have any idea about this.

Member Avatar for MktgRob

i request help on this topic: MSN, Google and Yahoo Directory currently ignore on of my clients website. None of the other search engines seems to have trouble.

The index.html is already in the three above. From there on, a cms is called through a frameset looking like this:

frame top= /cgi-bin/cms.pl?template=start
frame bottom= /cgi-bin/cms.pl?template=foot

These two pages are searchable as well (along with index.html) in Google, MSN and Yahoo, but none of the following pages.

So, i have a total of three pages , while some hundred are waiting since november 2002. The sites deeplinks look like this:

<a href = /cgi-bin/cms.pl?template=article //for categories
<a href = /cgi-bin/cms.pl?template=article&id=1234 //for articles

I used webmasterplan.de with their "this is how your site looks to engines" tool and found out that this tool requires "=" to be replaced by "%3d" (ascii opcode for equal signs). is this the same for google and company ?

Is your site completely build with a CMS? If so which one? One of the sites I maintain is completely done in CMS (and a really niche one at that) and I have no problem with this site being found by the SEs.

Be a part of the DaniWeb community

We're a friendly, industry-focused community of developers, IT pros, digital marketers, and technology enthusiasts meeting, networking, learning, and sharing knowledge.