Yelp Outing Businesses that Solicit Reviews

Original article on nytimes.com: http://www.nytimes.com/2012/10/18/technology/yelp-tries-to-h...

Seeing some of my favorite hole in the wall restaurants get bad reviews and seeing some bland and uninteresting (but good) places get great reviews on Yelp has made me remember Malcolm Gladwell's commentary on people's preferences for three distinct flavors of spaghetti sauce.

"If you sifted carefully through the data, though, you could find patterns, and Moskowitz learned that most people's preferences fell into one of three broad groups: plain, spicy, and extra-chunky"[1]

Which tells me that since Yelp tries to aggregate everyone's taste into one unified rating metric, the tastes from which Yelp ratings are derived are quite likely to be different from those of my own.

Thus I really can't trust Yelp ratings, even if there were no 'gaming' or 'extortion' going on in their reviews.

[1] http://www.gladwell.com/2004/2004_09_06_a_ketchup.html

Interesting point from the original nytimes article, that opens up a whole can of worms:

"What’s to stop someone from going and soliciting fake positive reviews for a competitor’s restaurant, in order for them to be publicly shamed?"

Well put! Don't forget to mention that Yelp has never turned a profit.

This is a really interesting point, unfortunately the counterpoint to this is that a complicated rating metric is not successful either. People don't want to spend ages filling in some huge questionnaire and if you start trying to introduce complicated systems of combination star ratings like the zagat ones for example, it just bamboozles both reviewers and readers, and still has the same issues as a simple 5 star rating does - it's too rough a tool and tries to decide in advance on a procrustean taxonomy into which all decisions are forced.

I think the most useful reviews are textual ones which set out the interests and prejudices of the writer (sometimes inadvertently), and elaborate on why they like or don't like this particular place/thing, which can give you clues about their lifestyle, tastes, and even whether they were hired by the company to write this review in the first place! I'm always suspicious of reviews which say something is great and then list several selling points as if they were read from a brochure - not many people write like that without prompting and it's a good way to find fakes.

Unfortunately another problem as outlined in that article is that even if you managed to completely represent the views of others, you'd find that many of them were simply uninteresting to you, as they are based on false premises or different tastes. That's where social rating systems become more important; where you can decide whose ratings you trust (not necessarily friends), and use those to make decisions - I still think you need the complexity of written reviews to really flesh out people's opinions though. We have a few ideas on this over at coolplaces, but have not yet had time to take them anywhere. Trust does require that you identify with a person's tastes, and also that you know they are real, so using a network of like-minded people built up over time can help mollify the concerns you have about yelp reviews.

Thanks for the great article linked, I've not had time to finish it but it really does cover this subject in depth from a different perspective.