Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How not to sort by average rating (2009)

evanmiller.org

Post reply on HN
Print view View original

11–20 of 27 posts

Re: How not to sort by average rating (2009)

#11

Post by ClassyJacket »

encom wrote: post #7
impure wrote: post #5

> Many people who find something mediocre will not bother to rate it at all; the act of viewing or purchasing something and declining to rate it contains useful information about that item’s quality. Not really sure, some people have different base likelihoods of rating something. I will rate apps when I see the app rating popup because it's simple and I know how important that app rating is. But the majority of peop…

>app rating popup If I see an app rating popup, I will rate the app 1 stars for annoying me with popups. I'm sick of rating things. I can't write an email to a company, without getting a follow-up email to rate my interaction with them. Same for telephone calls. Every time I charge my car on a public charger, the app will ask me to rate my charging experience. If I order a pizza, I get asked to rate it. Just... fuck…

Same. I have dozens of Play Store reviews that just say "Kept pestering me to rate, so here's your rating. One star."

I've tried to do this on Amazon for companies that attempted to bribe me for a positive review, but sadly Amazon doesn't allow you to mention review bribes in reviews, and they don't get posted.

Re: How not to sort by average rating (2009)

#12

Post by kuhewa »

nightowl_games wrote: post #8

I actually had this problem recently and I just sorted by absolute numbers of positive ratings and it worked pretty well. (You could only rate thumbs up/thumbs down). Nice to see a more mathy solution tho

I reckon a user might expect to see 100 up/1 down ranking higher than 1000 up/1000 down

Re: How not to sort by average rating (2009)

#13

Post by pjerem »

sahmeepee wrote: post #4
SubiculumCode wrote: post #3

Earlier quoted context omitted.

With this many prior discussions here, I guess the only question to ask is: Are any major sites using this method, or at least providing a way to combine sort with a filter items with under X ratings. If anything, it seems like the likes of Amazon are making it harder to sort and filter, using dark patterns to direct you to their preferred products.

Amazon's product search is largely useless, at least in the UK, now. It typically returns many items that don't include your search term, are outside the price range you specified and are not sorted in the order you asked for. The top half the list being "sponsored" items is just the enshittified cherry on the turd cake.

Also it’s now basically impossible to find the "usual" brands of a product if you don’t know them.

What I mean is, let’s say I want to buy an alarm clock. I don’t want a cheap one neither a luxury one. I just want to buy the medium alarm clock from usual brands you find in store but I don’t know, for a given type of product, which are the usual, reliable brands. Well, Amazon now being Aliexpress with a margin, it’s impossible. Just typing "alarm clock" in the search bar will submerge you with low quality products from temporary Chinese brands.

Re: How not to sort by average rating (2009)

#14

Post by add-sub-mul-div »

SubiculumCode wrote: post #3
dang wrote: post #2

Related: How not to sort by average rating (2009) - https://news.ycombinator.com/item?id=29200103 - Nov 2021 (82 comments) How Not to Sort by Average Rating (2009) - https://news.ycombinator.com/item?id=15131611 - Aug 2017 (156 comments) How Not to Sort by Average Rating (2009) - https://news.ycombinator.com/item?id=9855784 - July 2015 (59 comments) How Not To Sort By Average Rating - https://news.ycombinator.com/ite…

With this many prior discussions here, I guess the only question to ask is: Are any major sites using this method, or at least providing a way to combine sort with a filter items with under X ratings. If anything, it seems like the likes of Amazon are making it harder to sort and filter, using dark patterns to direct you to their preferred products.

You're right. Remember that Netflix contest where they solicited an algorithm that would make the most accurate movie predictions? It's a quaint vestige from a time before they figured out that the "algorithm" should optimize their profit rather than your media satisfaction, because those are not the same algorithm.

Re: How not to sort by average rating (2009)

#15

Post by romwell »

SubiculumCode wrote: post #3
dang wrote: post #2

Related: How not to sort by average rating (2009) - https://news.ycombinator.com/item?id=29200103 - Nov 2021 (82 comments) How Not to Sort by Average Rating (2009) - https://news.ycombinator.com/item?id=15131611 - Aug 2017 (156 comments) How Not to Sort by Average Rating (2009) - https://news.ycombinator.com/item?id=9855784 - July 2015 (59 comments) How Not To Sort By Average Rating - https://news.ycombinator.com/ite…

With this many prior discussions here, I guess the only question to ask is: Are any major sites using this method, or at least providing a way to combine sort with a filter items with under X ratings. If anything, it seems like the likes of Amazon are making it harder to sort and filter, using dark patterns to direct you to their preferred products.

> Are any major sites using this method, or at least providing a way to combine sort with a filter items with under X ratings

1. Reddit's default sorting algorithm was (and perhaps still is) based on this

2. Google's internal Q&A tool, Dory[1], used it to rank questions to ask to the leadership during the "open" mic sessions.

NOTE: please don't do the same mistake.

The math of the algorithm ensures that controversial questions never get asked, since the algorithm approximated the upvotes/downvotes ratio, not votes/views ratio.

The latter was not used because, IIRC, counting impressions on an internal site was too "technically challenging".

This, in my opinion, defies the entire point of an internal Q&A, which is to address hot topics before they spill out and become an issue.

(I don't think I'm spilling any tea talking about how Q&A in Google was run five years ago; there's no secret sauce there. It was natural to ask "which questions make it to the top", and the answer was the link to the article we're discussing. Google considered it to be fair).

[1] https://support.google.com/dory/answer/9254865?hl=en

Re: How not to sort by average rating (2009)

#16

Post by oli5679 »

Adding one to both the numerator and the denominator when calculating average ratings isn’t a terrible idea.

In situations where you’re estimating probabilities (like the average rating of an item based on user reviews), there is a Bayesian interpretation of this adjustment.

Beta distribution is a conjugate prior to the binomial distribution, meaning the update process has a posterior distribution is also a Beta distribution.

By adding one to the numerator (the number of positive reviews) and one to the denominator (the total number of reviews plus one), you’re effectively using a Beta(1,1) prior (uniform distribution).

This approach smooths the estimated average, especially for items with a small number of reviews. This is a useful regularisation, pulling extreme values towards the mean and reflecting the uncertainty inherent in limited data.

https://en.wikipedia.org/wiki/Beta_distribution

Re: How not to sort by average rating (2009)

#17

Post by remram »

nightowl_games wrote: post #8

I actually had this problem recently and I just sorted by absolute numbers of positive ratings and it worked pretty well. (You could only rate thumbs up/thumbs down). Nice to see a more mathy solution tho

This will give you popularity not rating, which is an interesting signal too.

Re: How not to sort by average rating (2009)

#18

Post by m463 »

pjerem wrote: post #13
sahmeepee wrote: post #4

Earlier quoted context omitted.

Amazon's product search is largely useless, at least in the UK, now. It typically returns many items that don't include your search term, are outside the price range you specified and are not sorted in the order you asked for. The top half the list being "sponsored" items is just the enshittified cherry on the turd cake.

Also it’s now basically impossible to find the "usual" brands of a product if you don’t know them. What I mean is, let’s say I want to buy an alarm clock. I don’t want a cheap one neither a luxury one. I just want to buy the medium alarm clock from usual brands you find in store but I don’t know, for a given type of product, which are the usual, reliable brands. Well, Amazon now being Aliexpress with a margin, it’s i…

Yeah, amazon has basically made brands worthless on their site.

I tried searching for a cordless handheld vacuum on amazon recently. All the 5.0 reviews are AI generated unverified purchases. And then just below that, lots of RANDOMWORD generic chinese "brands".

The reputable brands you might actually buy are hidden in the noise of it all.

Re: How not to sort by average rating (2009)

#19

Post by tiagod »

encom wrote: post #7
impure wrote: post #5

> Many people who find something mediocre will not bother to rate it at all; the act of viewing or purchasing something and declining to rate it contains useful information about that item’s quality. Not really sure, some people have different base likelihoods of rating something. I will rate apps when I see the app rating popup because it's simple and I know how important that app rating is. But the majority of peop…

>app rating popup If I see an app rating popup, I will rate the app 1 stars for annoying me with popups. I'm sick of rating things. I can't write an email to a company, without getting a follow-up email to rate my interaction with them. Same for telephone calls. Every time I charge my car on a public charger, the app will ask me to rate my charging experience. If I order a pizza, I get asked to rate it. Just... fuck…

At least for android apps, it's best to first rate 5 stars, then the lower score in Play store, as those pop-ups usually only redirect to play store for high ratings, to inflate their rank.

Re: How not to sort by average rating (2009)

#20

Post by Am4TIfIsER0ppos »

ClassyJacket wrote: post #11
encom wrote: post #7

Earlier quoted context omitted.

>app rating popup If I see an app rating popup, I will rate the app 1 stars for annoying me with popups. I'm sick of rating things. I can't write an email to a company, without getting a follow-up email to rate my interaction with them. Same for telephone calls. Every time I charge my car on a public charger, the app will ask me to rate my charging experience. If I order a pizza, I get asked to rate it. Just... fuck…

Same. I have dozens of Play Store reviews that just say "Kept pestering me to rate, so here's your rating. One star." I've tried to do this on Amazon for companies that attempted to bribe me for a positive review, but sadly Amazon doesn't allow you to mention review bribes in reviews, and they don't get posted.

> Amazon doesn't allow you to mention review bribes in reviews, and they don't get posted.

Imagine how obvious the problem would be if they allowed that to show up to the average customer.

Post reply on HN