Member Avatar for mitz

Hi guys
I thought my traffic was dropping a bit and looked at my site map and found 404 errors for every page I have.

All the usual pages were there but there were extra pages that didn't exist added into the sitemap. For example

404 error page
...

This does not exist on my website and what is the :80 in there?

Why is it looking for something that isn't there? There are at least 2000 of these urls..What can I do?

  • 2 Contributors
  • 2 Replies
  • 275 Views
  • 6 Days Discussion Span
  • Latest Post Latest Post by mitz

Dani AI

Generated

— this is a common symptom of how your site is handling missing pages rather than a sitemap generator bug. When a broken URL is rewritten or redirected to a single error page that includes the original path in the query string, each failed request becomes a unique URL (and can balloon into thousands). If the error page is served with a 200 OK (a "soft 404") instead of an actual 404 response, search engines will treat those generated URLs as real and report them in crawl/sitemap tools. already pointed out the odd :80 showing up — that is just the port being echoed back, not the real root cause.

Quick diagnostic checklist:

  1. Inspect raw server access logs to see the original requested paths and the referrers (this tells you whether those URLs are being requested from external sites or created internally).
  2. Check the HTTP status returned for one of the ugly URLs — it must return 404, not 200. Example check:
    curl -I "http://yourdomain/404error.htm?404;http://yourdomain:80/some/path"
  3. Verify your sitemap generator did not pick up those query-stringed error pages while crawling.

How to fix:

  • Make the 404 handler return a proper HTTP 404 status without issuing a client redirect that creates a new URL. Serve a local error document (server-side rewrite/transfer) rather than redirecting.
  • If using a CMS or ASP.NET/IIS, change the custom error behavior so the response code stays 404 (avoid Response.Redirect for missing pages).
  • Regenerate your sitemap after the fix and resubmit it to webmaster tools; remove the bad URLs from the sitemap or exclude query-stringed error pages.

After you implement the server-side fix and resubmit the cleaned sitemap, the bogus entries will drop out of crawl reports over time. If logs show external sites are the source, you can decide whether to leave those as 404, set up targeted 301s for legitimately moved content, or contact the linking sites.

Recommended Answers

All 2 Replies

Member Avatar for Dani

I'm confused what you mean by finding these pages in your sitemap? Is another site out there linking to these nonexistent pages? Adding :80 to the end of a domain name indicates to connect to the web server on port 80, which is the standard port that serves websites. It's implied by default when you connect to a website and don't specify a port (as we normally don't do).

Member Avatar for mitz

wow that is a great question.. I will see if another site is linking to those pages...
Because they really don't exist.

Be a part of the DaniWeb community

We're a friendly, industry-focused community of developers, IT pros, digital marketers, and technology enthusiasts meeting, networking, learning, and sharing knowledge.