Member Avatar for Dani

When we switched over from vBulletin to our own unique platform back in March 2012, one of the things that changed was the way that post quality score was calculated. Most members noticed their quality score had suddenly changed by a few points.

Since then, however, over the past year and a half, there have already been two separate instances of members complaining that an ill-willed member went on a downvoting spree with the sole intention of being malicious and affecting their quality score.

Our original formula for calculating post quality score was much better at softening the effects of this type of malice, and therefore I've decided to revert to the previous calculation.

Once again, you'll notice a change in your quality score today. I apologize for the confusion.

  • 4 Contributors
  • 5 Replies
  • 246 Views
  • 1 Hour Discussion Span
  • Latest Post Latest Post by JorgeM

Dani AI

Generated

Reverting the scoring formula was a sensible stopgap, but there are a few small, practical changes that will make quality scores both fairer and more meaningful over time — and that directly address the suggestions from (cutoff/recent emphasis) and (rankings that favor active contributors).

A robust, simple approach: compute a conservative score per post (Wilson lower bound for up/down votes) and aggregate those per-user with an exponential time decay so recent posts count more. That gives small, new posts less volatile influence while preserving lifetime contributions. Example pseudocode:

# per post:
n = up + down
phat = up / n if n>0 else 0.5
z = 1.96
wilson = (phat + z*z/(2*n) - z*sqrt((phat*(1-phat)+z*z/(4*n))/n)) / (1 + z*z/n)

age_years = (now - post_date).days / 365.25
half_life = 1.0  # adjust (0.5–2.0 years) to taste
weight = 2 ** (-age_years / half_life)

# per user:
user_score = sum(wilson * weight) / sum(weight)

Operational safeguards to add: detect and auto-reverse serial downvoting (many rapid downvotes against one author from a single account or tight cluster of accounts), rate-limit daily downvotes, and require a minimal reputation to downvote. Flag suspicious patterns for moderator review and keep an audit trail so reversals are transparent.

For UX and ranking: show both a lifetime score and a recent score (e.g., decayed/12-month window). Use the recent score for leaderboards so the top lists reflect active contributors while lifetime remains visible for historical recognition. Before any change, test on a copy of the DB, publish the exact algorithm and parameters, and provide a short appeals path so members understand and trust the system.

Recommended Answers

All 5 Replies

Member Avatar for JorgeM

I think this was worth your time. For some members, probably more than would admit, their scores are important to them.

Edited by JorgeM
Member Avatar for Dani

... says the person who has a 97% post quality score ;)

Member Avatar for mike_2000_17

The post quality score is definitely the best indicator for how worthy and awesome a member is. Shamelessly coming from another 97% scorer ;) (I'm sure JorgeM agrees with me!)

... says the person who has a 97% post quality score ;)

... says the person who has a puny 90% score ;)

Member Avatar for Ancient Dragon

Could you put a cut-off date for the calculations, for example only calculate for the most recent 3 or 4 years? That would de-emphasize the quality of very old posts.

Edited by Ancient Dragon
Member Avatar for JorgeM

Could you put a cut-off date for the calculations, for example only calculate for the most recent 3 or 4 years? That would de-emphasize the quality of very old posts.

I think that there shoulnt be a cut-off date for your individual scores, but I think that makes sense with regard to rankings.

For example, there are a lot of members that rank high in different categories, but havent contributed in years. What's the point in showcasing these members in the top 100 list. I think that your rank should be based upon your recent scores, say within the past 6 months to a year or so. This type of ranking would depict those members that are active and provide quality content to the site.

If this site is focusing on social community aspects, then i think that being social is being here and now, not about what happened 5 years ago....

Edited by JorgeM
Be a part of the DaniWeb community

We're a friendly, industry-focused community of developers, IT pros, digital marketers, and technology enthusiasts meeting, networking, learning, and sharing knowledge.