Member Avatar for paradox814
paradox814 1 Posting Whiz

I was working on my site, and was doing some mod_rewrites and realized that I very frequently forget to pass a 404 header if the requested page is invalid (and I found out that even this website is guilty of it!).

See this is what happens, when you use mod_rewrite technically any rule that matches is sent as OK (http status 200), even if the page shouldn't work. So for instance, while this one may work:
http://daniweb.com/techtalkforums/forum17.html
This one does not:
http://daniweb.com/techtalkforums/forum54517.html
but this last example does not send a 404 header and browsers and search engines alike assume that it is a valid page. And as we all know duplicate content from this mistake can hurt you on search engine ratings.

As soon as you find out the page is no good, you need to send the appropiate headers, for example:

header("HTTP/1.0 404 Not Found");
  • 1 Contributor
  • 0 Replies
  • 271 Views

Dani AI

Generated

@gparadox814 Good catch — mod_rewrite itself only maps URLs; the HTTP status sent to the client is whatever the final handler emits. If your rewrite always points to a script that returns 200 even for missing resources, browsers and search engines will treat those URLs as real (soft‑404s) and you get duplicate/garbage pages indexed.

Prefer application-level checks. Let rewrites funnel requests to your controller, then verify the requested resource (DB lookup, filesystem check) and send a real 404 when it’s not found. For modern PHP you can do something like:

if (! $forum) {
    http_response_code(404);
    include '404.php';
    exit;
}

If you must handle it at the rewrite layer, mod_rewrite can return proper codes directly. Use the R flag to send a 404 or the G flag for 410, and only apply those rules when you can reliably determine “not found” with a regex or filesystem tests:

RewriteEngine On

# only rewrite if no real file or dir exists
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d

# return 404 for obviously impossible IDs (example)
RewriteRule ^techtalkforums/forum([0-9]{6,})\.html$ - [R=404,L]

# permanently gone
RewriteRule ^old-section/ - [G,L]

Quick testing and best practices: check status with curl -I https://example.com/nonexistent, watch server logs, and avoid custom 404 pages that themselves return 200. Use ErrorDocument 404 /404.php (local path) so Apache preserves the 404 status, prefer 410 for permanently removed resources, and 301 for moved content. Regex tricks to guess “nonexistent” are brittle — let your application do the definitive check whenever possible.

Be a part of the DaniWeb community

We're a friendly, industry-focused community of developers, IT pros, digital marketers, and technology enthusiasts meeting, networking, learning, and sharing knowledge.