Member Avatar for dado.d
dado.d 0 Newbie Poster

I was crawling http://www.imenik.hr (more specific: http://www.imenik.hr/imenik/trazi/1/ptt:51551.html) and it was working. I used PHP Symfony crawler, I even used proxies. I was testing it locally so I tried without proxies.

I added 5 seconds of sleep time between each requests, I tried random sleep between requests, I tried 10 seconds. After only few requests it gave me this page http://www.imenik.hr/imenik/captcha.html

I tried incognito and it gave me the same page. I guess it ties my crawling with IP address. How does it even recognizes those requests as non human?

  • 1 Contributor
  • 0 Replies
  • 26 Views
Be a part of the DaniWeb community

We're a friendly, industry-focused community of developers, IT pros, digital marketers, and technology enthusiasts meeting, networking, learning, and sharing knowledge.