Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

NPS doesn’t say anything particularly useful

cranberryblog.substack.com

Post reply on HN
Print view View original

41–50 of 83 posts

Re: NPS doesn’t say anything particularly useful

#41

Post by trynewideas »

abeyer wrote: post #37
hn_throwaway_99 wrote: post #9

I see proposals against NPS like this from time to time, but they always tend to ignore the underlying science that went into making NPS in the first place, primarily that there was a very strong correlation between the calculated NPS score and business performance. Also, I find that many of these articles have somewhat of a caricaturish view of how NPS is used in the real world. "It's not 'oh no, our NPS score is do…

Like sibling comment said, I'd love to see actual science behind it. > a caricaturish view of how NPS is used in the real world. "It's not 'oh no, our NPS score is down, let's hire more people to focus on customer 'happiness'" That sounds exactly like what responses at many companies would be. There's often much nuanced discussion about how/when/why to adopt certain metrics, but once they are they become sacrosanct a…

Agreed. The first line of the article says specifically for "companies and their investors to stop using Net Promoter Score (NPS) as a KPI", and NPS doesn't provide a measurement that directly supports specific actionable steps; KPIs are supposed to be about planning for actionable, measurable steps toward a clear business outcome.

(And I have plenty of opinions on how useless KPIs have been in practice for me and the places/teams I've worked. They're hard to get right unless everyone in management is on the same page about what a clearly defined business goal is, and what an actionable step is, and what a strong metric for determining whether the action met the goal is.)

Re: NPS doesn’t say anything particularly useful

#42

Post by aagha »

hn_throwaway_99 wrote: post #9

I see proposals against NPS like this from time to time, but they always tend to ignore the underlying science that went into making NPS in the first place, primarily that there was a very strong correlation between the calculated NPS score and business performance. Also, I find that many of these articles have somewhat of a caricaturish view of how NPS is used in the real world. "It's not 'oh no, our NPS score is do…

> As a software engineer I love reading our NPS reports, because they always have great tidbits of info that are often difficult to deduce just by looking at standard analytics.

What you love is the customer feedback--this can be accomplished in multiple ways and you don't need NPS for it.

Re: NPS doesn’t say anything particularly useful

#43

Post by trynewideas »

madeofpalk wrote: post #5

My "problem" with NPS is when essential services, which I have next to no choice over using, ask me if i would recommend them to others. No, IRS, I would never recommend your service to friends or family. That's not really indicative of anything.

One company I worked at used NPS internally to measure employee satisfaction and holy hell did they not enjoy seeing what that looked like.

Re: NPS doesn’t say anything particularly useful

#44

Post by mdorazio »

riversflow wrote: post #16
mdorazio wrote: post #4

I didn't see discussed here the main drivers you might want NPS vs. outcome- (and company) specific metrics: 1) it's harder to game and/or bullshit. This is a massive issue with surveying customers and then using their responses to feed back into internal performance goals & rewards. 2) because it has grown to be such a common customer experience metric, it's possible to benchmark against others, which isn't really p…

> it's harder to game and/or bullshit. . . it's possible to benchmark against others, I don't have access to A/B testing of a large business to understand how things affect responses. But I certainly intuit that there is a significance in when/how you ask someone for feedback. e.g. If you ask your question directly after the checkout screen you'll get a significantly different response than if you survey people rando…

Because if you're a big company you don't run your own surveys - you pay one of the big CX companies like Maritz, Medallia, InMoment, etc. to run them for you or for smaller feedback not requiring a full management system you hire one of the 100+ research firms to do it for you. Because these firms run surveys for tons of clients, they generally have standard practices for how & when feedback is collected and are willing to share industry-standard benchmarks for things like NPS. I worked with two of the names above on a 2-year stint with a major auto manufacturer that sent over a million surveys a year and NPS comparisons were one of many tools used to get meaning out of responses.

Re: NPS doesn’t say anything particularly useful

#45

Post by phillipcarter »

hn_throwaway_99 wrote: post #9

I see proposals against NPS like this from time to time, but they always tend to ignore the underlying science that went into making NPS in the first place, primarily that there was a very strong correlation between the calculated NPS score and business performance. Also, I find that many of these articles have somewhat of a caricaturish view of how NPS is used in the real world. "It's not 'oh no, our NPS score is do…

The "science" is quite dubious: https://articles.uie.com/net-promoter-score-considered-harmf...

The reason why NPS exists is because it's an easy number to calculate that sounds convincing, and manager-types love numbers without having to think too hard about how they got them.

Re: NPS doesn’t say anything particularly useful

#46

Post by itsdrewmiller »

rkk3 wrote: post #24

> (The most irritating — and by far the most common — reason companies seem to measure NPS is that it’s standardized, but more importantly that everyone else does it. Which is fine for one-time-use benchmarking, but not a good basis for internal KPIs.) Except the alternative isn't to stop tracking NPS the alternative is to use unstandardized customer survey metric's, this is worse because you have no true industry co…

https://www.qualtrics.com/experience-management/customer/wha... is another standardized alternative, that I suspect would suffer less from the problems in the article.

Re: NPS doesn’t say anything particularly useful

#47

Post by itsdrewmiller »

phillipcarter wrote: post #45
hn_throwaway_99 wrote: post #9

I see proposals against NPS like this from time to time, but they always tend to ignore the underlying science that went into making NPS in the first place, primarily that there was a very strong correlation between the calculated NPS score and business performance. Also, I find that many of these articles have somewhat of a caricaturish view of how NPS is used in the real world. "It's not 'oh no, our NPS score is do…

The "science" is quite dubious: https://articles.uie.com/net-promoter-score-considered-harmf... The reason why NPS exists is because it's an easy number to calculate that sounds convincing, and manager-types love numbers without having to think too hard about how they got them.

That article just links to the wikipedia page. The main criticism paper[1] reasonably successfully debunks NPS as uniquely valuable, but more or less shows it as of comparable value to other measures of customer satisfaction.

1: https://web.archive.org/web/20200716065914/https://pdfs.sema...

Re: NPS doesn’t say anything particularly useful

#49

Post by throwawaysea »

This post is correct about two things - first, that NPS does not give you direction on how to act. Second, that statistical realities make sampling approaches like NPS very noisy. I don’t think either discounts the concept of NPS itself. “Bad statistical work” is going to be bad regardless of what metric we’re talking about. And a metrics being too abstract to action doesn’t make it useless - at least not any more so than something like free cash flow. Yes, connecting the dots to some outcome is difficult. But it’s a pulse on the business, and if done right, that has some inherent value.

Re: NPS doesn’t say anything particularly useful

#50

Post by kqr »

Lazare wrote: post #36

You know what would be vastly better than NPS? Ask people every so often how they're feeling about the product. Let them pick a thumbs up or a thumbs down. If they pick thumbs down, say "we're sorry you hear that. Please let us know what's bothering you", and display a little text box. Carefully read every comment you receive. Asking unhappy users for feedback is a great idea. The 0 to 10 scale and - especially - the…

> Asking unhappy users for feedback is a great idea. The 0 to 10 scale and - especially - the bizarre calculation NPS does with it - is statistical malpractice

I disagree. On the contrary, it is good survey practise to ask yes/no questions with a scale, to get information on intensity of opinion. (Often this is a scale 1--7 ranging from "strongly disagree" through "neutral" to "strongly agree", but there are other alternatives of which NPS is one.)

Then you might think only the bottom half of the range would formally be considered "detractors", and that would (as far as the evidence goes) be accurate. However, we're not looking for how many people are detractors today -- we want to know how many people are at risk of becoming detractors in the near future. That's what makes NPS a somewhat leading metric. And that's also why you see the sleight of hand that counts answers up to 6 as detractors.

I don't know what "mathematically capable" means in your context, nor what "NPS supports claim" in your experience, but the number absolutely makes sense.

Post reply on HN