Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Problem with Feedback

theatlantic.com

Post reply on HN
Print view View original

11–20 of 45 posts

Re: The Problem with Feedback

#11

Post by nitwit005 »

jasonshen wrote: post #2

This is a frustrating piece to read as a startup founder and product manager because I don't think the author understand how much companies already do think about the things established in the piece. > "If thumbs-ups or ratings on a five-point scale are not automatically useful, what kind of feedback would be? Finely tuned feedback that targets the system it’s meant to regulate will always surpass a barrage of angry…

The more questions you ask on a survey, the more the results tend to be garbage. Either people will give up on it, or they'll fill in everything with the same value.

Re: The Problem with Feedback

#12

Post by orev »

One issue I have is when feedback is clearly being collected with intent to manipulate, like with a cable company. They ask you to specifically rate the individual person, and you know that the person sitting in a call center all day might lose their job because of a bad rating. You might give a good rating even if you had bad service (or at least you’ll give a 5 instead of a 3). Then the company turns around and advertises that they have the best customer satisfaction ratings, etc. when in reality everyone hates them, but doesn’t want to blame it on the poor person stuck in the call center.

Re: The Problem with Feedback

#13

Post by jetrink »

iaabtpbtpnn wrote: post #9
a13n wrote: post #6

I've been working on a startup in the user feedback space [1] full-time for the past 3.5 years, and love to see the topic of user feedback on Hacker News. Uber doesn't want your feedback because they care about your ideas/opinions. They want it because it's critical to their business. Put yourself in their shoes. They have 75 million users and 3 million drivers. A critical part of growth is retention: not bleeding th…

I give Uber drivers 5 stars unless they did something egregiously horrible, because I know that if I give 4 stars, and enough people do the same, that driver will be banned from the platform, even though most people would say that 4 out of 5 ain't bad. Similarly, if I ever actually take the time to fill out a receipt survey from a retail store, I just give full marks across the board, because I know that otherwise so…

It's as if these companies were an extraterrestrial intelligence and mathematics our only common language. What does a 4 express? The other absurdity is that the only context for these rating is the value that appears under a driver's portrait. One person is 4.67 driver, while another is a 4.89. Now rate your driver using the numbers: 1, 2, 3, 4 or 5.

Re: The Problem with Feedback

#14

Post by kaycebasques »

a13n wrote: post #6

I've been working on a startup in the user feedback space [1] full-time for the past 3.5 years, and love to see the topic of user feedback on Hacker News. Uber doesn't want your feedback because they care about your ideas/opinions. They want it because it's critical to their business. Put yourself in their shoes. They have 75 million users and 3 million drivers. A critical part of growth is retention: not bleeding th…

> love to see the topic of user feedback on Hacker News

I'm also very interested in the topic, but in a slightly different domain. I'm on a mission to get "was this page helpful?" feedback widgets on all technical documentation across the web. Recently I just helped the Kubernetes folks [1] add it to their docs site.

One of my high-level goals is to get the docs community to aggregate all of this data in order to benchmark typical ratings for common types of documents. E.g. what's the typical rating for a tutorial? I also want us to start actively experimenting in order to validate what kinds of changes lead to meaningful improvements in documentation. E.g. adding examples to conceptual overviews improves the rating by 5% (not real numbers).

Find me on the interwebz if you'd like help implementing one of these systems in your docs, or want to join this project.

[1] https://github.com/kubernetes/website/pull/11037

Re: The Problem with Feedback

#15

Post by hugh-avherald »

iaabtpbtpnn wrote: post #9
a13n wrote: post #6

I've been working on a startup in the user feedback space [1] full-time for the past 3.5 years, and love to see the topic of user feedback on Hacker News. Uber doesn't want your feedback because they care about your ideas/opinions. They want it because it's critical to their business. Put yourself in their shoes. They have 75 million users and 3 million drivers. A critical part of growth is retention: not bleeding th…

I give Uber drivers 5 stars unless they did something egregiously horrible, because I know that if I give 4 stars, and enough people do the same, that driver will be banned from the platform, even though most people would say that 4 out of 5 ain't bad. Similarly, if I ever actually take the time to fill out a receipt survey from a retail store, I just give full marks across the board, because I know that otherwise so…

I'm surprised Uber doesn't add a remark like "Your rating for will be compared to your ratings of other drivers." beneath the rating. This could (a) encourage people to give 1-5 star ratings more uniformly and (b) discourage constantly giving 5-star ratings.

Depending on how much they wanted to get more uniformly distributed ratings, they could (a) make the rider rating depend on the uniformity of the ratings given or (b) just use a comparison rating ("Was this better or worse than your previous trip?").

Re: The Problem with Feedback

#16

Post by stuartaxelowen »

taeric wrote: post #8
jasonshen wrote: post #2

This is a frustrating piece to read as a startup founder and product manager because I don't think the author understand how much companies already do think about the things established in the piece. > "If thumbs-ups or ratings on a five-point scale are not automatically useful, what kind of feedback would be? Finely tuned feedback that targets the system it’s meant to regulate will always surpass a barrage of angry…

"back test data to outcomes" sounds scarily like p-hacking. That is, it seems highly likely you will still not find anything actually meaningful. You are just guaranteeing that you will be able to find something. This is especially true if you have retention numbers, already. Why bother asking if I was happy with a purchase, if I never buy again anyway. Note that I am sympathetic. There is every chance that I will bu…

Not so - the business is looking for real ROI, and thus will not be satisfied with topics that are "in the noise". Also, you create the measurement that establishes a problem before you apply the treatment (create new feature X or fix bug Y), and then verify after treatment that the desired affect was achieved.

Re: The Problem with Feedback

#17

Post by gonehome »

a13n wrote: post #6

I've been working on a startup in the user feedback space [1] full-time for the past 3.5 years, and love to see the topic of user feedback on Hacker News. Uber doesn't want your feedback because they care about your ideas/opinions. They want it because it's critical to their business. Put yourself in their shoes. They have 75 million users and 3 million drivers. A critical part of growth is retention: not bleeding th…

Uber's feedback is the most frustrating because I can't rate in the only instances that I'd care to.

- Driver calls and asks where I'm going then cancels ride

- Driver drives up to me and asks where I'm going then cancels ride

- Driver drives around aimlessly for 40min trying to get me to cancel and then cancels ride

- Driver for Uber eats picks up food, steals it, then cancels ride

In all of these instances the cancellation removes the ability to rate - these are the cases when I'd want to rate zero stars.

If the person picks me up and takes me where I need to go I'll rate them five stars, in the worst case I just won't rate them.

Re: The Problem with Feedback

#18

Post by stuartaxelowen »

I think the author has it wrong, what the author sees as a "barrage of angry or ecstatic reviews" is actually a rich array of personal experiences waiting to be interpreted.

Traditionally, this interpretation has been done wholly by humans, and was a long and laborious task. However, we can now do much of the manual work with NLP [1] - discovering topics of concern and measuring the prevalence and sentiment. This lets us ask the dataset useful questions, like "What issue is angering customers most?", and "What features should we be talking more about?"

It's a move towards a much, much wider sampling of customer experiences (compared to focus groups), and I think it should be celebrated for empowering companies to make products/services that consumers really, really like.

[1] https://taggit.io

Re: The Problem with Feedback

#19

Post by mark-r »

I have trouble completing those surveys - I want to be fair and accurate, but it's simply not possible with subjective impressions and no guidelines. Life is too short for such angst; I generally refuse surveys when they're presented without a second thought.

Re: The Problem with Feedback

#20

Post by exclusiv »

gonehome wrote: post #17
a13n wrote: post #6

I've been working on a startup in the user feedback space [1] full-time for the past 3.5 years, and love to see the topic of user feedback on Hacker News. Uber doesn't want your feedback because they care about your ideas/opinions. They want it because it's critical to their business. Put yourself in their shoes. They have 75 million users and 3 million drivers. A critical part of growth is retention: not bleeding th…

Uber's feedback is the most frustrating because I can't rate in the only instances that I'd care to. - Driver calls and asks where I'm going then cancels ride - Driver drives up to me and asks where I'm going then cancels ride - Driver drives around aimlessly for 40min trying to get me to cancel and then cancels ride - Driver for Uber eats picks up food, steals it, then cancels ride In all of these instances the canc…

I've seen this quite often too! In my experience it happened more with Lyft and when I hailed a ride from the airport.

I live pretty close to the airport (6-20 mins) so the sneaky drivers that see the fare will call me and confirm my destination. I confirm and then they just sit in their waiting spot indefinitely, hoping I cancel.

Understanding their strategy, I walk down to the lower level with the app still open and take a cab home. They see me physically moving (haha) and have choice but to cancel. I leave the app open even when I get home.

I even had one driver listed as hearing impaired who called me up to confirm my destination. He had no trouble hearing and confirmed he would be right there to get me (only to just wait it out so I would hopefully cancel!)

Next time - leave the app open and use another service or grab a taxi. They'll see you move on the map and you'll waste their time since they were trying to waste yours. :)

Also - I always tip about double the norm for any driver that picks me up and doesn't have a bad attitude because I know they would have preferred a longer trip. I shouldn't have to do this, but I'm a business owner and I don't mind rewarding those that do the right things.

Post reply on HN