Member Avatar for Naor

Do you have experience with similarweb. It's a company that collect data about websites. do you know how they count bot traffic? I mean if you have a website with 500k montly visits? DO similarweb filter bot traffic from this number? Any input on this matter will help. Thanks!!

  • 3 Contributors
  • 3 Replies
  • 295 Views
  • 1 Month Discussion Span
  • Latest Post Latest Post by becketharper585

Dani AI

Generated

Quick expert note for : and raised the right issues. Rather than trying to prove a single cause, run a few short checks that will tell you whether a Similarweb “500k” estimate is likely inflated by automated traffic and where the difference is coming from.

  • Align time windows and definitions first — make sure you compare the same month/day range and account for Similarweb’s timezone and release cadence. Similarweb Data Methodology FAQs. (support.similarweb.com)

  • Inspect engagement and distribution signals: sudden unexplained spikes, unnatural pages/visit or session-duration numbers, very concentrated geography, or extreme browser/device uniformity are common red flags to investigate. Similarweb’s own guidance lists these checks for spotting fake traffic. How to spot fake traffic. (similarweb.com)

Deeper verification (practical): use server/CDN logs as the ground truth—these capture requests that client-side scripts miss. From logs you can run reverse-DNS/IP-range checks to verify major crawlers, and flag headless-browser patterns or unusually high request rates from single IP blocks. There are recent guides showing how to separate “training” crawlers from real-time search traffic and why logs matter for this work. Server-log guide. IP verification tips. (tryzenith.ai)

Quick experiment and mitigation: pick a suspicious IP range or UA, apply temporary rate-limiting or a bot-management rule, then compare week-over-week changes in your logs and in Similarweb. For scalable filtering, consider a bot-management layer (many use heuristics + JS checks + ML at the edge). If alignment with third-party estimates is business-critical, ask Similarweb about direct-measurement options. Cloudflare bot-detection overview. (developers.cloudflare.com)

Treat third‑party traffic tools as directional; use logs + short experiments to decide whether that 500k figure is plausible for your reporting or needs correction.

Recommended Answers

All 3 Replies

Member Avatar for Dani

Similarweb is only able to estimate the amount of traffic you have by collecting data on a small sample of user browsing habits (that they have access to) and then extrapolating that. Here is an article about the different ways they collect data: https://support.similarweb.com/hc/en-us/articles/4912885128337-How-does-Similarweb-get-data-

The only way they would know exactly how much traffic you have is if you connected your site’s Google Analytics account to their platform. You can get more information about that at https://support.similarweb.com/hc/en-us/articles/208420125-Connecting-Google-Analytics-and-Similarweb

Since Google Analytics does not count bot traffic, SimilarWeb would not know about it either.

Member Avatar for becketharper585

SimilarWeb is a website that provides data and analytics on website traffic. It uses a combination of data sources to provide information on website traffic.
SimilarWeb uses a multi-step process to determine bot traffic. Several methods are used to identify bot traffic, such as user-agent, browser fingerprints, and IP addresses. Machine learning algorithms are also used to identify patterns in the data that indicate bot traffic. Once the bot traffic is identified, it is removed from the data set before it is reported to clients.

commented: thanks! do you have any source to verify it? +0
Member Avatar for becketharper585

Similar Web Ltd is a web-analytic company that deals with web traffic and performance. You may visit the website and go through their documentation and understand their data sources, where you can detect and filter out the bot traffic. Or you may contact SimilarWeb directly through email or phone and ask them about the techniques they use to detect and filter out the bot traffic.
Also, you can compare the data from SimilarWeb with other analytic tools and internal data.
This is what I know, hope it helps.

Be a part of the DaniWeb community

We're a friendly, industry-focused community of developers, IT pros, digital marketers, and technology enthusiasts meeting, networking, learning, and sharing knowledge.