Member Avatar for afsdfasdf654654

So, can you explain to me, why your website/host is running a website scraper abusing the server and resources of another online community that is (unlike your own) providing a free AND ad-free service to its users?
Every hour at x:25 your server is requesting a page hosted on our server.
Is that the kind of behaviour you want to "teach" the people here? To use online scrapers that ignore the robots.txt?
Maybe I should contact your hoster and file an abuse claim.

dtpp commented: pics/logs or go away..... +0

Recommended Answers

All 11 Replies

Member Avatar for afsdfasdf654654

169.55.25.105 - - [03/Oct/2015:07:25:09 +0000] "HEAD <path_removed>?show=calendar HTTP/1.1" 200 20 "-" "-" "-" "-" - - 12677
169.55.25.105 - - [03/Oct/2015:08:25:04 +0000] "HEAD <path_removed>?show=calendar HTTP/1.1" 200 20 "-" "-" "-" "-" - - 43943
169.55.25.105 - - [03/Oct/2015:09:25:04 +0000] "HEAD <path_removed>?show=calendar HTTP/1.1" 200 20 "-" "-" "-" "-" - - 12910
169.55.25.105 - - [03/Oct/2015:10:25:14 +0000] "HEAD <path_removed>?show=calendar HTTP/1.1" 200 20 "-" "-" "-" "-" - - 35234
169.55.25.105 - - [03/Oct/2015:12:25:08 +0000] "HEAD <path_removed>?show=calendar HTTP/1.1" 200 20 "-" "-" "-" "-" - - 34194
169.55.25.105 - - [03/Oct/2015:13:25:04 +0000] "HEAD <path_removed>?show=calendar HTTP/1.1" 200 20 "-" "-" "-" "-" - - 19884
169.55.25.105 - - [03/Oct/2015:14:25:09 +0000] "HEAD <path_removed>?show=calendar HTTP/1.1" 200 20 "-" "-" "-" "-" - - 38838
169.55.25.105 - - [03/Oct/2015:15:25:04 +0000] "HEAD <path_removed>?show=calendar HTTP/1.1" 200 20 "-" "-" "-" "-" - - 14583
169.55.25.105 - - [03/Oct/2015:16:25:04 +0000] "HEAD <path_removed>?show=calendar HTTP/1.1" 200 20 "-" "-" "-" "-" - - 45777
169.55.25.105 - - [03/Oct/2015:17:25:09 +0000] "HEAD <path_removed>?show=calendar HTTP/1.1" 200 20 "-" "-" "-" "-" - - 11875
169.55.25.105 - - [03/Oct/2015:18:25:04 +0000] "HEAD <path_removed>?show=calendar HTTP/1.1" 200 20 "-" "-" "-" "-" - - 35265
169.55.25.105 - - [03/Oct/2015:19:25:04 +0000] "HEAD <path_removed>?show=calendar HTTP/1.1" 200 20 "-" "-" "-" "-" - - 19448
169.55.25.105 - - [03/Oct/2015:20:25:09 +0000] "HEAD <path_removed>?show=calendar HTTP/1.1" 200 20 "-" "-" "-" "-" - - 62902

Hi,

Firstly, you can choose to disable ads from within your member settings.

Secondly, we don't have any cron jobs running 25 after the hour that attempt to request any external pages.

We do have a cron job that attempts to fetch external pages that people link to within posts to ensure that the link is valid (so that we aren't linking to broken pages). However, it certainly doesn't re-check the same link every hour, and like I said, none of the cron jobs we have related to links run 25 after the hour.

Can you please send me a private message with the full URL so that I can investigate this further please?

Member Avatar for afsdfasdf654654

No, I cant send you a pm. Because my account is sandboxed or whatever the reason is.
And 169.55.25.105 is clearly your www4 host. Which requested the page in question again at 21:25 and 22:25 UTC.

I also searched for our domain on your site and only found 1 match from 2 years ago. Which wasn't a link anway.

commented: www4 means its virtual host +0

I've sent you a private message. You should be able to reply to it.

Member Avatar for diafol

@your-site-sucks - this really is a PM issue, not general feedback. Please conduct this off-forum with Dani as requested. Thanks.

We've been going back and forth. Seems to be a bug on my side but I can't get to the bottom of it.

Member Avatar for afsdfasdf654654

Still havent found anything?

I still haven't, I'm really sorry. I can't find anything in our logs that is showing that we are attempting to retrieve your site, and the databse query is not picking up on your URL either.

Member Avatar for afsdfasdf654654

If you cant find the cause, can you maybe just remove the url from that post? That would at least solve the issue for our site.

Perhaps one or both of you could add a blocking rule to your respective firewalls or a HOSTS entry to redirect to 127.0.0.1

If you cant find the cause, can you maybe just remove the url from that post? That would at least solve the issue for our site.

From my perspective, the issue related to your site is already [temporarily] resolved: you added our IP address to your firewall, and the problem is mitigated for the time being. I am much more concerned with why it is happening, how widespread the issue is, and potentially how many other websites are being affected by this bug.

Be a part of the DaniWeb community

We're a friendly, industry-focused community of developers, IT pros, digital marketers, and technology enthusiasts meeting, networking, learning, and sharing knowledge.