The Problem with Feedback
theatlantic.com
The Problem with Feedback
1–10 of 45 posts
Re: The Problem with Feedback
#2This is a frustrating piece to read as a startup founder and product manager because I don't think the author understand how much companies already do think about the things established in the piece.
> "If thumbs-ups or ratings on a five-point scale are not automatically useful, what kind of feedback would be? Finely tuned feedback that targets the system it’s meant to regulate will always surpass a barrage of angry or ecstatic reviews. Rather than trumpeting the desirability of all feedback, apps and review sites should pursue only the information that is crucial for making the system work better."
It's hard to know what data points correlate to making the system work better until after the fact — so developers generally have to "over ask" so you can back test data to outcomes.
Probably the best criticism of the proliferation around feedback is that it doesn't always make the customer's experience better, it just creates better conditions for the company to make more money.
Re: The Problem with Feedback
#3"Think about it: The proliferation of ratings systems doesn’t necessarily produce a better restaurant or hotel experience. Instead, it homogenizes the offerings, as people all go to the same top-rated establishments. Those places garner ever more reviews, bouncing them even farther up the list of results. Rather than a quality check, feedback here becomes a means to bland sameness."
The problem is, these rating lack context. Consider a pop song with a gazillion streams on YouTube. How many of those were "unemployed" 15 year olds pretending to be sick and staying home from school? Without some sense of context those gazillion likes have little meaning.
And of course, baring some drastically awful experience, there's confirmation bias. The truth is most people aren't objective enough to admit they might have made a bad choice. Of course it was the best ____ ever! It was my ____. I bought it.
Even so, there are always idiots who will order pizza at a place that specializes in curry, or curry from a place that does Mexican. Yes, it's on the menu. But that's doesn't mean you should order it and then complain about it.
Re: The Problem with Feedback
#4This is a frustrating piece to read as a startup founder and product manager because I don't think the author understand how much companies already do think about the things established in the piece. > "If thumbs-ups or ratings on a five-point scale are not automatically useful, what kind of feedback would be? Finely tuned feedback that targets the system it’s meant to regulate will always surpass a barrage of angry…
I think the author's point is much more around intent. Asking for feedback is one thing. Having a plan in place to do something with it is another. It's so easy to bake in a feedback loop but that doesn't translate to any actual change. NPS is popular not because it's necessarily useful, but because it's easy to understand and track. I worked in market research for years and our NPS remained flat because we didn't have the right means of sharing and implementing the feedback we gathered. The same themes came up over and over again over the years and yet we never changed our actual product. This is the point the author is touching on.
Re: The Problem with Feedback
#5This is a frustrating piece to read as a startup founder and product manager because I don't think the author understand how much companies already do think about the things established in the piece. > "If thumbs-ups or ratings on a five-point scale are not automatically useful, what kind of feedback would be? Finely tuned feedback that targets the system it’s meant to regulate will always surpass a barrage of angry…
re: "doesn't always make the customer's experience better,"
Does it really if it does? Or is people __believing__ it does good enough? If your comfortable with the latter then such feedback will create confirmation bias. Which the product dev team (also being human) is also going to fall victim to.
Re: The Problem with Feedback
#6I've been working on a startup in the user feedback space [1] full-time for the past 3.5 years, and love to see the topic of user feedback on Hacker News.
Uber doesn't want your feedback because they care about your ideas/opinions. They want it because it's critical to their business.
Put yourself in their shoes. They have 75 million users and 3 million drivers. A critical part of growth is retention: not bleeding the existing user base you have, by offering a quality service.
So how do they identify undesirable drivers to maintain the safety and quality of their service? And in a scalable way...
You ask users to give drivers a review out of 5 stars. Chances are, if multiple people give a driver 1 star, others will too. It's a pretty good proxy for driver quality.
I can definitely see how this feels "dehumanizing" and like you're a "cog in the system". You are a cog in the system. You're a cog in their automated system for keeping their service safe and high quality.
There are different kinds of user feedback, for different reasons. One kind is user interviews, where you want to hear people's ideas and opinions for how you can do better. This is pretty human. Reviews are pretty automated, and just a different kind of "feedback".
If you run a consumer company (as opposed to B2B), it isn't practical to conduct user interviews on even 1% of your entire user base. So it isn't surprising that giving feedback to consumer companies tends to be a pretty negative experience.
[1] https://canny.io
Re: The Problem with Feedback
#7I've been working on a startup in the user feedback space [1] full-time for the past 3.5 years, and love to see the topic of user feedback on Hacker News. Uber doesn't want your feedback because they care about your ideas/opinions. They want it because it's critical to their business. Put yourself in their shoes. They have 75 million users and 3 million drivers. A critical part of growth is retention: not bleeding th…
I want to be a cog in 95% of my interactions. I only try to extract meaning from a few of my relationships.
Re: The Problem with Feedback
#8This is a frustrating piece to read as a startup founder and product manager because I don't think the author understand how much companies already do think about the things established in the piece. > "If thumbs-ups or ratings on a five-point scale are not automatically useful, what kind of feedback would be? Finely tuned feedback that targets the system it’s meant to regulate will always surpass a barrage of angry…
"back test data to outcomes" sounds scarily like p-hacking. That is, it seems highly likely you will still not find anything actually meaningful. You are just guaranteeing that you will be able to find something.
This is especially true if you have retention numbers, already. Why bother asking if I was happy with a purchase, if I never buy again anyway.
Note that I am sympathetic. There is every chance that I will buy something again simply because the exit survey reminded me of your existence. There is also the chance having an exit survey will make me not want to come back. :)
Re: The Problem with Feedback
#9I've been working on a startup in the user feedback space [1] full-time for the past 3.5 years, and love to see the topic of user feedback on Hacker News. Uber doesn't want your feedback because they care about your ideas/opinions. They want it because it's critical to their business. Put yourself in their shoes. They have 75 million users and 3 million drivers. A critical part of growth is retention: not bleeding th…
I give Uber drivers 5 stars unless they did something egregiously horrible, because I know that if I give 4 stars, and enough people do the same, that driver will be banned from the platform, even though most people would say that 4 out of 5 ain't bad. Similarly, if I ever actually take the time to fill out a receipt survey from a retail store, I just give full marks across the board, because I know that otherwise some poor underpaid single mom is gonna have to "be accountable" for my responses to an uncaring corporate drone. On the other hand, a person giving a 1 to an Uber driver or bad marks across the board on a retail survey is probably unduly pissed off, and upon further reflection (or exposure to the consequences of their score) even they might not agree they suffered a truly 1-worthy experience. So my question to you is, how do you distinguish signal from noise?
Re: The Problem with Feedback
#10This is why Netflix eliminated the star ratings -- it was too hard to figure out the degree of difference inherent in four vs 5 stars, plus it was a manual step.
And to the author: kybernetes (κυβερνήτης) is the steersman of a boat, not a governor.