Live data from Hacker News

If your customers don't talk, NPS is a vanity metric

elliotcsmith.com

21–30 of 57 posts

Re: If your customers don't talk, NPS is a vanity metric

#21
post #9

NPS, Myers-Briggs Type Indicator, and Astrology are all things that seem scientific, get a lot of attention, and are junk.

Astrology has some validity if you dig deep enough:

* when should we plant this crop?

* how do I generate a random number if literally any human bias is worse than a random choice?

* were you the oldest (or youngest) in your class at school?

Re: If your customers don't talk, NPS is a vanity metric

#22
Every time you get one of those surveys rank them at zero, then add "Net Promoter Score is a flawed vanity metric and shouldn't be used for business purposes" in the comment box. Sometimes I link the Wikipedia NPS "Criticism" section as well.

Most places don't care about the results from an actual customer service perspective. The above gets crickets, not even an auto responder.

For companies that do care (tiny startups, mostly) I've gotten IMMEDIATE personal email responses from CEOs and founders asking what they can fix for a zero NPS. That's a great place to link the criticism section if not done previously, and to provide useful, raw feedback on what you love/hate about their products.

Re: If your customers don't talk, NPS is a vanity metric

#25

In stead of asking “would you reccomend”, shouldn’t you rely on “how did you learn about us?”, or “why did you buy our product”, the first one being the most reliable, as it is a neutral question about a fact.

Would you recommend is probably easier to track as an OKR. You can keep asking it every quarter for new data to pat (or smack) yourself.

Re: If your customers don't talk, NPS is a vanity metric

#26

I’m inferring that NPS here refers to “Net Promoter Score” [0]. Presumably within his niche, the author can assume his readers already know that. [0] https://en.wikipedia.org/wiki/Net_promoter_score

I would have expected everyone to have heard of it by now. I first saw it mentioned on HN in 2017 or so. Good reminder that there's always newer folks around and to define the terms!

That has nothing to do with newer folks, more with interest in certain topics.

Re: If your customers don't talk, NPS is a vanity metric

#27
post #22

Every time you get one of those surveys rank them at zero, then add "Net Promoter Score is a flawed vanity metric and shouldn't be used for business purposes" in the comment box. Sometimes I link the Wikipedia NPS "Criticism" section as well. Most places don't care about the results from an actual customer service perspective. The above gets crickets, not even an auto responder. For companies that do care (tiny start…

This tanks the evaluation of any individuals you interacted with while dealing with the company, which can impact their pay or even push them towards getting laid off for low performance. So I'd advise caution when applying this particular idea, since many employers use these surveys to decide who to fire.

Re: If your customers don't talk, NPS is a vanity metric

#28
post #22

Every time you get one of those surveys rank them at zero, then add "Net Promoter Score is a flawed vanity metric and shouldn't be used for business purposes" in the comment box. Sometimes I link the Wikipedia NPS "Criticism" section as well. Most places don't care about the results from an actual customer service perspective. The above gets crickets, not even an auto responder. For companies that do care (tiny start…

This tanks the evaluation of any individuals you interacted with while dealing with the company, which can impact their pay or even push them towards getting laid off for low performance. So I'd advise caution when applying this particular idea, since many employers use these surveys to decide who to fire.

That's "negotiate with terrorists" logic. I'm not going to pretend to take some company's bullshit seriously because they implicitly threaten to fire their employees at random if I don't.

(I do advocate for laws against arbitrary firings and encourage employees to unionise and/or move to jurisdictions with strong labour laws).

Re: If your customers don't talk, NPS is a vanity metric

#29
> I need you to understand that people don't have conversations where they randomly recommend operating systems to one another

Maybe you are in the wrong crowd, but the crowd that does do recommendations on operating systems it is rarely windows that is recommended

Re: If your customers don't talk, NPS is a vanity metric

#30
post #3

I would go a step further: NPS is a garbage metric. First, you start by assuming your customers can even reasonably ascertain their likelihood to recommend. They can't; there are people who answer 10 but will never recommend, and there are people who answer 0 but already have and will again. Next, you assume your customers are idiots and don't know how an 11-point scale works by adjusting the midpoint: Instead of 5,…

Based on practical experience, NPS is garbage because:

1) Even with stable mean and median, NPS tends to vary month over month, at least for my B2B settings where samples are probably much smaller than for B2C. Then, management goes nuts because of very subtle shifts in the distribution caused by NPS' arbitrary aggregation into promoters, neutrals, detractors. Of course, often investors are married to NPS, so educating management does not solve the problem.

2) NPS varies unreasonably across cultures. We used to say, somewhat tongue-in-cheek, that NPS is a US-centric metric, where things are either amazing or awful (with little space in between). E.g., in northern/central Europe, an 8 can be pretty amazing.

Post reply on HN