Member Avatar for bigginge

A customer whose site I maintain rang this morning saying his website traffic had suddenly increased to getting about 20 times as many hits as usual in the last few days, and asked if this was something to worry about.
I'm not very knowledgeable about this side of things but found his access files where the ip address '' came up a lot. This wasn't present in week old access files. I googled the address and it came up in a page about bots versus browsers.
Can anyone shed any light on all this, please? The full line in the access log reads:
- - [27/Jul/2009:00:01:26 +0200] "GET / HTTP/1.1" 200 12469 www.sitename.co.uk "-" "MSNPTC/1.0 (compatible; MSIE 6.0; Windows NT 5.2; MyIE2; .NET CLR 1.1.4322; .NET CLR 1.0.3705)" "-"

  • 6 Contributors
  • 6 Replies
  • 205 Views
  • 1 Week Discussion Span
  • Latest Post Latest Post by MidiMagic

Dani AI

Generated

A concise, practical triage and mitigation note for and the thread:

Start by triaging, not reacting. Confirm whether the spike is actually degrading the site (CPU, memory, bandwidth, error‑rates) and collect the patterns: top client IPs, top user‑agents, the most requested paths and referrers, and the per‑minute rate for any suspect host. Quick log commands (Apache/Nginx combined log format):

# top IPs
awk '{print $1}' /var/log/nginx/access.log | sort | uniq -c | sort -nr | head

# top user agents
awk -F'"' '{print $6}' /var/log/nginx/access.log | sort | uniq -c | sort -nr | head

# per-minute hit-rate for a suspect IP
grep 'SUSPECT_IP' /var/log/nginx/access.log | cut -d'[' -f2 | cut -d']' -f1 | cut -d: -f1-2 | uniq -c | sort -nr | head

Validate suspected crawlers before blocking. User‑agent strings are trivial to fake; perform a reverse DNS + forward-confirmed DNS lookup (the standard verification method used by major crawlers) or use the official verification tools where available. This prevents accidentally blocking legitimate search/indexing traffic. (developers.google.com)

If the traffic is abusive or high‑frequency, prefer throttling and temporary bans over an immediate permanent IP block. Server‑level rate limiting (example NGINX) and automated banning systems (fail2ban) are effective first steps; for Apache consider mod_evasive/mod_security. Example NGINX limit and a simple fail2ban jail:

# nginx (http block + server block example)
limit_req_zone $binary_remote_addr zone=perip:10m rate=5r/s;

server {
    location / {
        limit_req zone=perip burst=10 nodelay;
    }
}

# fail2ban (jail.local snippet)
[nginx-badbots]
enabled = true
port = http,https
filter = nginx-badbots
logpath = /var/log/nginx/access.log
maxretry = 2
bantime = 86400

Use these controls while you investigate; tune thresholds to avoid false positives. (blog.nginx.org)

Escalate if the incident is volumetric (bandwidth/pipe saturated): contact the host/ISP and consider a CDN/WAF that can absorb or scrub attack traffic. Be cautious with meta tags or blunt robots rules — they affect indexing and crawl behavior if applied incorrectly. Link/share‑driven one‑off spikes, scripted email clients, aggressive crawlers, and deliberate DoS all behave differently; correlate logs before making permanent blocks. (developers.cloudflare.com)

Notes tied to existing replies: ’s block suggestion is a valid quick test but should be temporary and verified; ’s suggestion about shared clients/links is a plausible explanation for a transient spike; and rightly flagged DoS/scraping risk; and ’s bot‑blocking warning about index impact is important to keep in mind.

Recommended Answers

All 6 Replies

Member Avatar for thefandango

Try adding the IP address to the blocklist in your server/cpanel and see if traffic drops.

Member Avatar for Nattynooster

It looks like someone may be trying to DoS the website (random attack) to make the server go down, if you have unlimited bandwidth and it is not slowing the site down then you do not need to worry but as thefandango suggested it may be an idea to add the IP to the blocklist

Member Avatar for almostbob

whois
_________________
07/28/09 00:40:11 IP block
Trying at ARIN
Trying 207.46.119 at ARIN

OrgName: Microsoft Corp
OrgID: MSFT
Address: One Microsoft Way
City: Redmond
StateProv: WA
PostalCode: 98052
Country: US

NetRange: -
CIDR:
NetName: MICROSOFT-GLOBAL-NET
NetHandle: NET-207-46-0-0-1
Parent: NET-207-0-0-0-0
NetType: Direct Assignment
NameServer: NS1.MSFT.NET
NameServer: NS5.MSFT.NET
NameServer: NS2.MSFT.NET
NameServer: NS3.MSFT.NET
NameServer: NS4.MSFT.NET
Comment:
RegDate: 1997-03-31
Updated: 2004-12-09

RTechHandle: ZM39-ARIN
RTechName: Microsoft
RTechPhone: +1-425-882-8080
RTechEmail:

OrgAbuseHandle: ABUSE231-ARIN
OrgAbuseName: Abuse
OrgAbusePhone: +1-425-882-8080
OrgAbuseEmail:

OrgAbuseHandle: HOTMA-ARIN
OrgAbuseName: Hotmail Abuse
OrgAbusePhone: +1-425-882-8080
OrgAbuseEmail:

OrgAbuseHandle: MSNAB-ARIN
OrgAbuseName: MSN ABUSE
OrgAbusePhone: +1-425-882-8080
OrgAbuseEmail:

OrgNOCHandle: ZM23-ARIN
OrgNOCName: Microsoft Corporation
OrgNOCPhone: +1-425-882-8080
OrgNOCEmail:

OrgTechHandle: MSFTP-ARIN
OrgTechName: MSFT-POC
OrgTechPhone: +1-425-882-8080
OrgTechEmail:

# ARIN WHOIS database, last updated 2009-07-27 20:00
# Enter ? for additional hints on searching ARIN's WHOIS database.
_______
Microsoft hotmail servers:
someone has posted a link to your site in an email the went to an imap folder. when the mail is viewed and imap refreshes (sometimes as small as 1second depending on settings) the link reloads the page.

some think that refreshing the mail folder causes mail to be received faster, but being able to refresh the folder at ridiculous intervals is not efficient
it will be a temporary spike until the mail user no longer uses that link to access the site

Member Avatar for MidiMagic

A cracker may be trying to overload the site to exploit vulnerability.

Member Avatar for PopeJareth

I don't know if this is related but, once several Google bots were constantly ambushing my site and were the cause of it going down.

Do you guys think that could be a possible cause. All we had to do was add a meta tag to stop the bots

Member Avatar for MidiMagic

Stop the bots, and you are no longer indexed on search engines.

Be a part of the DaniWeb community

We're a friendly, industry-focused community of developers, IT pros, digital marketers, and technology enthusiasts meeting, networking, learning, and sharing knowledge.