The genius is adding some previous scores. I always struggle to rate stuff fairly without anything obvious to compare it with.
Brewing a Better Rating System
11–20 of 38 posts
Re: Brewing a Better Rating System
#12I think main reason sliders aren't used is that users find it too troublesome, hence up/down and 5 stars are mainly used. I remember from my pys class that a 7 point rating system is best. But the 5 stars' simplicity and ubiquity probably trumps the benefits gained by a 7 point system. I think the best compromised is a 5 star UI implemented as 6 points by allowing 0 point assignments.
Re: Brewing a Better Rating System
#13I love that the sliding meter shows tick marks for your previous rankings of other teas - the UI reflects that your judgement of a particular tea is relative to your other experiences.
I suspect under closer analysis ones scoring breaks down to be inconsistent - "in retrospect I like tea X better than Y but not as much as Z, but I rated Z lower than Y because I didn't like it as much as P which had a higher rating", if you follow.
Re: Brewing a Better Rating System
#14The problem with this system is in the sorting. The list of "Highest Rated" teas is dominated by results where 1 person rated the tea 100.
Steepster should use a Bayesian average (http://en.wikipedia.org/wiki/Bayesian_average) so that the uncertainty of a small number of ratings is reflected in the sorting.
Re: Brewing a Better Rating System
#15The problem with this system is in the sorting. The list of "Highest Rated" teas is dominated by results where 1 person rated the tea 100. Steepster should use a Bayesian average ( http://en.wikipedia.org/wiki/Bayesian_average ) so that the uncertainty of a small number of ratings is reflected in the sorting.
Yeah, sorting is an issue we're still looking at (and is still very much in transition considering the new rating system). Appreciate the suggestion! We'll add it to our list of potential solutions.
Re: Brewing a Better Rating System
#16This is very cool thank you. I posted earlier today about the need for a better rating system than the five star system. I'm really glad to see you working on a great solution. Thanks!
Re: Brewing a Better Rating System
#17I love that the sliding meter shows tick marks for your previous rankings of other teas - the UI reflects that your judgement of a particular tea is relative to your other experiences.
I suspect under closer analysis ones scoring breaks down to be inconsistent - "in retrospect I like tea X better than Y but not as much as Z, but I rated Z lower than Y because I didn't like it as much as P which had a higher rating", if you follow.
It seems that, pairwise, it is pretty easy to decide. Maybe one could go further than this and eliminate the absolute scale all together (at least at rating time).
I'm imaging a UI which asks you to pick a favorite among the item you are viewing and one similar item. You could stop there, or ask repeatedly with new comparison items until the viewed item's position on the absolute scale is unambiguous. The user could provide some rating data with just a single binary decision, but some ajax-y fade out/in of another pair could enable further ratings if they desired.
Re: Brewing a Better Rating System
#18The problem with this system is in the sorting. The list of "Highest Rated" teas is dominated by results where 1 person rated the tea 100. Steepster should use a Bayesian average ( http://en.wikipedia.org/wiki/Bayesian_average ) so that the uncertainty of a small number of ratings is reflected in the sorting.
Yeah, sorting is an issue we're still looking at (and is still very much in transition considering the new rating system). Appreciate the suggestion! We'll add it to our list of potential solutions.
Start everything off with a single 50-point score. That way one person ranking it 100 will bump it up to 75, the next to 83, and so on. Such a system would cause teas that have more people upvoting them to rank higher than those that just happen to have one or two good opinions.
Re: Brewing a Better Rating System
#19Earlier quoted context omitted.
Mike, great job on this. Very informative. I have 2 questions for you: 1. is your slider from the jQuery UI or other js framework? 2. in regards to combating the 4.3 dilemma, have you found the average ratings on steepster to be lower? maybe its too early to tell, but I'd love to see some sort of curve on your ratings distribution in a future post ... thanks
Thanks, callmeed. 1. Yep, slider implementation is jQuery UI. 2. It is too early to tell, but we're definitely planning to share a follow up. As mentioned in the post, we had a simple thumbs up/down for ratings and were seeing a greater than 90% positive average, so we were definitely experiencing that bias. Just today, albeit with a much too small sample size, we're starting to see a more diverse mix of averages. We…
It'd also be interesting to know if the number of ratings decrease or increase. I wonder if your users will find the added granularity a nuisance or an incentive.
Re: Brewing a Better Rating System
#20About time someone created a better rating systems. (Stars RIP). It will be interesting to see if users like and use the slider.