Yeah, sorting is an issue we're still looking at (and is still very much in transition considering the new rating system). Appreciate the suggestion! We'll add it to our list of potential solutions.
Start everything off with a single 50-point score. That way one person ranking it 100 will bump it up to 75, the next to 83, and so on. Such a system would cause teas that have more people upvoting them to rank higher than those that just happen to have one or two good opinions.
That's a special case of a particular sort of Bayesian average.
Thanks, callmeed. 1. Yep, slider implementation is jQuery UI. 2. It is too early to tell, but we're definitely planning to share a follow up. As mentioned in the post, we had a simple thumbs up/down for ratings and were seeing a greater than 90% positive average, so we were definitely experiencing that bias. Just today, albeit with a much too small sample size, we're starting to see a more diverse mix of averages. We…
It'd also be interesting to know if the number of ratings decrease or increase. I wonder if your users will find the added granularity a nuisance or an incentive.
It will be interesting. It's important to note the nature of our community and whom we expect to contribute. Generally, we're geared toward a more passionate user who we find to be more than willing to contribute at this level of granularity. So we've made the choice to cater toward their needs while still trying to remain accessible.
But, this is a good point, and I think an important one to consider when evaluating the mechanic that works best for your community/site.
Something else you could think about in a rating system like this would to instead of using generic faces, you could associate each with a common tea that most tea lovers have tried.
The notches kind of do this but theres always the risk of someone rating there first tea 80, then deciding subsequent teas after are better so they need to be rated higher, when the first one should have been more around 60.
I suspect under closer analysis ones scoring breaks down to be inconsistent - "in retrospect I like tea X better than Y but not as much as Z, but I rated Z lower than Y because I didn't like it as much as P which had a higher rating", if you follow.
It seems that, pairwise, it is pretty easy to decide. Maybe one could go further than this and eliminate the absolute scale all together (at least at rating time). I'm imaging a UI which asks you to pick a favorite among the item you are viewing and one similar item. You could stop there, or ask repeatedly with new comparison items until the viewed item's position on the absolute scale is unambiguous. The user could…
If you do that, the Elo rating system is a good place to start algorithm-wise.
I suspect under closer analysis ones scoring breaks down to be inconsistent - "in retrospect I like tea X better than Y but not as much as Z, but I rated Z lower than Y because I didn't like it as much as P which had a higher rating", if you follow.
It seems that, pairwise, it is pretty easy to decide. Maybe one could go further than this and eliminate the absolute scale all together (at least at rating time). I'm imaging a UI which asks you to pick a favorite among the item you are viewing and one similar item. You could stop there, or ask repeatedly with new comparison items until the viewed item's position on the absolute scale is unambiguous. The user could…
Taking this idea further, one could add multiple orthogonal axes (eg sweetness, bitterness, after-taste etc). Then you could rate each tea against others on each axis - either in a star-slider, scatter plot or on several individual sliders.
This would allow you to rate teas against each other based on several aspects, and possibly allow recommendations based on how other people have rated teas - eg 'I want a tea that is not-too-sweet, a little bitter with a lingering aftertaste'
Then again, it does add more features / visual clutter and possibly complicates things for people...
I think main reason sliders aren't used is that users find it too troublesome, hence up/down and 5 stars are mainly used. I remember from my pys class that a 7 point rating system is best. But the 5 stars' simplicity and ubiquity probably trumps the benefits gained by a 7 point system. I think the best compromised is a 5 star UI implemented as 6 points by allowing 0 point assignments.
For those who marked me down, would you please comment on reason? I'm getting tired of spending my time commenting and getting disapproval without reason. I don't think down votes should be on disagreements; it should be on spamish, childish, or comments that does not add anything to the topic. My main point is that sliders aren't used much because they are too troublesome for a typical user. If you disagree with that, please add your opinion. I'm not trying to take anything away from the author. In fact, I think it's an ingenious idea. But I usually dislike repeating "wow, cool" comments since so many others have done so already. It's part of my DRYness kicking in.
I love that the sliding meter shows tick marks for your previous rankings of other teas - the UI reflects that your judgement of a particular tea is relative to your other experiences.
Exactly. I think the tick marks are the killer feature here. Great idea!
An interesting idea might be to let the user modify a tick mark in retrospect when scoring a new tea.
I think main reason sliders aren't used is that users find it too troublesome, hence up/down and 5 stars are mainly used. I remember from my pys class that a 7 point rating system is best. But the 5 stars' simplicity and ubiquity probably trumps the benefits gained by a 7 point system. I think the best compromised is a 5 star UI implemented as 6 points by allowing 0 point assignments.
Got anything to backup the 'users find sliders troublesome' assertion? I’d agree with you if you said developers, but I haven’t heard about any user problems.
Hi, I'm Mike from Steepster. We thought we'd share our new ratings system we just deployed with HN as we think it's relevant for products with customer reviews, ratings, etc. It's our attempt to combat the 4.3 dilemma (discussed here recently: http://news.ycombinator.com/item?id=883890 ). Background: Steepster is a community site for tea drinkers to share their tasting notes, get recommendations, and discover new tea…
Very clever. Have you considered making the slider non-linear (the distance on the slider between Yuck and Meh is smaller than between Good and Awesome)? If most people are going to rate their tea somewhere between Good and Awesome, it allows more of the slider to be used.
Have you received enough ratings with this new interface to know if my assumption of ratings being clustered is accurate?