Member Avatar for Dani

I can use my Apache server logs to figure out how many pulls there were of my RSS feed each day ... and therefore can track usage trends. But this just isn't enough for me. Is there ANY way that I can figure out whether my feeds are being pulled from a news syndicator, from a website syndicating my feeds, from 1 big site or from 100 little sites? etc etc

  • 3 Contributors
  • 2 Replies
  • 114 Views
  • 3 Months Discussion Span
  • Latest Post Latest Post by pulse

Dani AI

Generated

Server logs give raw pull counts, but they don’t always reveal who’s really republishing your items: many readers and syndicators proxy requests, omit Referer headers, or share hosting blocks. As and noted, feed-analytics/wrapping services are one option; below are practical, implementable techniques you can use yourself to separate a single big consumer from many small ones.

  • Improve and parse logs: record %{Referer}i and %{User-Agent}i so you can spot repeat user-agents or empty referers. Apache’s logging docs explain formats and options (https://httpd.apache.org/docs/2.4/logs.html). Example format to capture both fields: LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined.
  • Group by network owner: map client IPs to ASNs or run reverse DNS to see whether requests come from one organization (a syndicator/CDN) or many distinct hosts. Team Cymru’s IP-to-ASN service is useful for bulk lookups (https://team-cymru.org/Services/ip-to-asn/).
  • Use distinct feed endpoints per partner: create per-syndicator URLs (prefer path-based, e.g. /feeds/partner-name) so pulls are trivially attributable. Note some proxies strip query strings, so path tokens are more reliable.
  • Find republishers and measure renders: search a unique sentence in quotes to discover sites republishing your content (Google advanced search tips: ). For actual item views rather than pulls, embed a 1x1 tracking image in full-content feeds (web beacon) and count image requests — many clients block images, so this measures views, not pulls (https://en.wikipedia.org/wiki/Web_beacon).
  • Consider push/webhook flows: implementing WebSub gives more explicit subscriber/hub metadata than passive polling (https://www.w3.org/TR/websub/). Also track conditional GETs (If-None-Match/304) to estimate how many subscribers poll only for changes (see Referer header behavior at MDN: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Referer).

Recommendation: for definitive attribution, use per-partner feed URLs plus IP/ASN grouping, and supplement with targeted content searches and optional beacon counting. Beware privacy rules and know that Referer/User-Agent data can be missing or forged, so combine multiple techniques for reliable conclusions.

Recommended Answers

All 2 Replies

Member Avatar for etechsupport

Do you have used Feedburners? it allows you to make your feed more friendly by using their Smartfeed system you can also make your feed browser friendly.

Member Avatar for pulse

The only one that I have come across is Feedburner. :) WHy not give em a try?

Be a part of the DaniWeb community

We're a friendly, industry-focused community of developers, IT pros, digital marketers, and technology enthusiasts meeting, networking, learning, and sharing knowledge.