A log file paradox 227,000 hits this month and no visitors

[tcomc]

These are the facts
for a real life case study.

unique ip address 160,000 this month
logged with a basic but accurate program
written in house
approx 8000 unique ip's a day .
227,000 hits this month
aw & in house agree .
+ 5000 zipped software downloads from site resources .

google analytics shows less than 10 visitors a day
real time analytics showing about this number
awstats
82 visitors a day
99.4 of all traffic is bookmarked/direct
reflecting the fact presumably that our freely available software integrates with a web site
unknown browsers/os in excess of 90% of all traffic
add to favorite 118% more than the number of visitors shown

can anyone throw some light on this puzzle

[markchicobaby]

Sure its not spammers/leechers? Set up the domain with leech protection in Apache.

Its probably not too hard for spammers to spoof IP addresses, or download via a bot net of IP addresses.

Are you sure those IPs are unique? Run them through sort -u or similar. Don't trust the tools, get the logs yourself and go through them.

Finally, have you examined the logs yourself, to see what the stats programs are seeing. You should be able to trace whether a user is accessing the site or just sucking down zips, in which case you've got bots coming at ya.

M

[Dani]

Is your in house tracking software JavaScript based? I'd be willing to bet that it's counting bots and search engine spiders, which Google Analytics doesn't do. (Yes, they'd be many unique IPs).

[markchicobaby]

Is your in house tracking software JavaScript based? I'd be willing to bet that it's counting bots and search engine spiders, which Google Analytics doesn't do. (Yes, they'd be many unique IPs).

Not that many. Would spiders really generate 8,000 unique IPs per day?

[Dani]

As I am typing this, there are 636 Yahoo! slurp spiders, each with different IPs, crawling DaniWeb right this second. And I'm not even going to get into Googlebot!

[markchicobaby]

What about the 5000 zip downloads? Spiders should be ignoring these?

[tcomc]

Could be but
aw stats shows less than 5000 hits
can be attributed to spiders .
google shows next to no traffic and
no time spent on site .
Aw shows an average of 18 minutes 30 % 3000 visits in excess of 1 hour .
real time can not find these visits .
Raw log files 29 MB
showing lots of activity far more than aw
or google .

one set of stats is saying this site is healthy
another that it has tumbleweed blowing through it ! .