Member Avatar for whitestream6

In my Apache log yesterday, I noticed these strange errors:

127.0.0.1 - - [17/Apr/2011:15:02:36 +0100] "GET /icons/blank.gif HTTP/1.1" 200 487 "http://localhost/" "msnbot/1.1 (+http://search.msn.com/msnbot.htm)"
127.0.0.1 - - [17/Apr/2011:15:02:36 +0100] "GET /icons/layout.gif HTTP/1.1" 200 616 "http://localhost/" "msnbot/1.1 (+http://search.msn.com/msnbot.htm)"
127.0.0.1 - - [17/Apr/2011:15:02:36 +0100] "GET /icons/text.gif HTTP/1.1" 200 569 "http://localhost/" "msnbot/1.1 (+http://search.msn.com/msnbot.htm)"
127.0.0.1 - - [17/Apr/2011:00:02:36 +0100] "GET /icons/folder.gif HTTP/1.1" 200 564 "http://localhost/" "msnbot/1.1 (+http://search.msn.com/msnbot.htm)"
127.0.0.1 - - [17/Apr/2011:15:02:36 +0100] "GET /icons/image2.gif HTTP/1.1" 200 650 "http://localhost/" "msnbot/1.1 (+http://search.msn.com/msnbot.htm)"
127.0.0.1 - - [17/Apr/2011:15:02:36 +0100] "GET /icons/unknown.gif HTTP/1.1" 200 585 "http://localhost/" "msnbot/1.1 (+http://search.msn.com/msnbot.htm)"
127.0.0.1 - - [17/Apr/2011:15:02:36 +0100] "GET /icons/compressed.gif HTTP/1.1" 200 1379 "http://localhost/" "msnbot/1.1 (+http://search.msn.com/msnbot.htm)"
127.0.0.1 - - [17/Apr/2011:15:02:36 +0100] "GET /favicon.ico HTTP/1.1" 200 39421 "-" "msnbot/1.1 (+http://search.msn.com/msnbot.htm)"

How could this have happened, and what should I do to prevent this?
I know for a fact I don't have any /icons folder directory, only the favicon.ico file, so this is puzzling me.
Figuring out what caused it that is the problem here.

Thanks! :)

Edited by whitestream6 because: n/a
  • 2 Contributors
  • 1 Reply
  • 766 Views
  • 2 Days Discussion Span
  • Latest Post Latest Post by jino

Dani AI

Generated

Short answer: those GETs are almost always the Apache “icons” served for server-generated directory listings — they come from an Alias, not from a file under your vhost DocumentRoot, so a missing /icons folder in www/html does not prove the files are not being served. Apache’s autoindexing (AddIcon / AddIconByType / DefaultIcon) and the Alias mechanism are what produce those /icons/* GIF responses. (httpd.apache.org)

Why the log shows 127.0.0.1 and a “msnbot” user‑agent: the access log’s %h is the TCP client address, so 127.0.0.1 means the connection originated on localhost (local health check, a monitoring script, or a front‑end proxy like nginx/varnish that connects on loopback). If a proxy is used it will often mask the real client IP unless X‑Forwarded‑For is passed and Apache is configured to restore it (mod_remoteip or equivalent). User‑agent strings like “msnbot/1.1” are trivial to fake; the correct way to verify a Microsoft crawler is reverse‑then‑forward DNS confirmation as described by Bing. (httpd.apache.org)

Practical checklist (quick, actionable):

  • Search logs for other msnbot entries with non‑local IPs to see whether this is local only.
  • Look for an Alias /icons/ or the httpd-autoindex.conf include in your Apache config and disable/comment it if you do not want those icons exposed. (tenable.com)
  • If using a proxy, make it send X‑Forwarded‑For and enable mod_remoteip so logs show real clients. (httpd.apache.org)
  • Verify suspicious crawlers with reverse+forward DNS (Bing’s guidance) before blocking. (blogs.bing.com)
  • Use robots.txt for polite crawlers but rely on firewall/mod_security rules when you must actively block bad actors; do not indiscriminately block 127.0.0.1 (it can break health checks). For robots.txt behavior and limits see the search engine guidance. (developers.google.com)

Notes: — absence of an on‑disk /icons directory is expected in many distributions because Apache serves those images from a packaged icons directory. ’s pointer toward checking crawler/robots behavior is sensible; combine that with the config/log checks above.

Be a part of the DaniWeb community

We're a friendly, industry-focused community of developers, IT pros, digital marketers, and technology enthusiasts meeting, networking, learning, and sharing knowledge.