Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

If your customers don't talk, NPS is a vanity metric

elliotcsmith.com

Post reply on HN
Print view View original

41–50 of 57 posts

Re: If your customers don't talk, NPS is a vanity metric

#41

Post by blitzar »

mcintyre1994 wrote: post #34

I always find this metric bizarre when companies ask about it. Everything is in either a category of things I’m not going to bother talking about, or I’ve already recommended it to people who might care, or I’ve already anti-recommended it to those people. Those are 0, 10, 0 I guess. I’m never like “well I haven’t mentioned it yet but there’s a 60% chance I’ll recommend it at some point” From when I worked at a compa…

I did not realise they meant it literally.

Would I recommend a product to others? Yes. Does it ever come up in conversation? No. Do I go around telling random people about this product? No.

Re: If your customers don't talk, NPS is a vanity metric

#42

Post by Simon_O_Rourke »

I work in a company where they try NPS both with customers and staff. Both of these NPS results were available in very fine grained (anonymous) reports... That is until quite recently.

Previously we had access to all the freeform comments, such as "as a customer we need feature X", or "as a staff member we want to see more transparency around Y".

Today, after a few particularly turblent quarters including layoffs, all we get to see are summarized versions of the staff NPS.

Vanity project indeed.

Re: If your customers don't talk, NPS is a vanity metric

#43

Post by blitzar »

shoo wrote: post #40
ggm wrote: post #6

I work in a not for profit functional monopoly space. A member body distributing resources according to policy. I can get why we do satisfaction scores, but NPS never made sense to me. Like, does the justice system do NPS for family law cases? It's not like you can get divorced in a 7-11 by the clerk, and even mediation is a process inside the model so NPS in a non competitive self-regulated monopoly.. what does it e…

On a scale of 0 to 10, how likely are you to recommend the use of NPS as a metric to a fictitious competitor in your not for profit functional monopoly space?

7

Re: If your customers don't talk, NPS is a vanity metric

#44

Post by nonameiguess »

forgotoldacc wrote: post #8
FearNotDaniel wrote: post #2

I’m inferring that NPS here refers to “Net Promoter Score” [0]. Presumably within his niche, the author can assume his readers already know that. [0] https://en.wikipedia.org/wiki/Net_promoter_score

I never knew this was an actual "thing." Like, I've seen these for years. It feels like 5 seconds after I open any given website a "rate us on 1-5" popup shows up right in the middle of the screen. I assumed it was just some thing that's automatically thrown in just to annoy users and has no practical purpose, like the cookie warnings (that are ignored regardless of what you select), the email spam requests (which no…

I suppose I should have known they're being used for something, but I tend to not answer these because they don't seem answerable. I had a surgery a few weeks back and had an e-mail asking me to rate the experience afterward. Anesthesia results in most memories of the direct experience not existing, so I can't rate that. My satisfaction with the surgery itself depends on the ultimate outcome of it, which I can't possibly know within a day.

It's the same thing with regular product purchase. You get these "would you recommend?" or "would you purchase again?" solicitations either immediately after receipt of the product or within a few days, or solicitations to go leave a review somewhere. How am I supposed to rate something I've owned for somewhere between minutes and days and have barely used? Many of my most disappointing purchases seemed like great things when they first arrived or I first put them together, but after prolonged usage they showed serious flaws or simply stopped working.

Re: If your customers don't talk, NPS is a vanity metric

#45

Post by ggm »

shoo wrote: post #40
ggm wrote: post #6

I work in a not for profit functional monopoly space. A member body distributing resources according to policy. I can get why we do satisfaction scores, but NPS never made sense to me. Like, does the justice system do NPS for family law cases? It's not like you can get divorced in a 7-11 by the clerk, and even mediation is a process inside the model so NPS in a non competitive self-regulated monopoly.. what does it e…

On a scale of 0 to 10, how likely are you to recommend the use of NPS as a metric to a fictitious competitor in your not for profit functional monopoly space?

"I wish all my fictitious competitors would do this"

Re: If your customers don't talk, NPS is a vanity metric

#46

Post by smcin »

beAbU wrote: post #35
smcin wrote: post #12

Earlier quoted context omitted.

Essentially it says few customers will move the needle on bringing you new business, only the ones who rave wildly about you. Hence, "Customer obsession" as Amazon's Bezos used to put it. Since NPS and changes in NPS is measured by surveys, not attributing individual sales, it's a KPI that's gameable (especially when used on its own); near-impossible to tell whether "improving" NPS in some existing-customer segment f…

There's irony in bringing up Amazon here: From what I've seen, the thing most people rave about when it comes to Amazon is it's customer service. Everything else is pretty mid: the products are poor quality, sometimes counterfeit and if you are not careful way overpriced. Amazon is not selling customer service as a separate product, yet that's the product that most customers really rave about, even though they don't…

I said "used to put it". In the early days of Amazon, it totally differentiated itself from other online and brick-and-mortar sellers of books(/CDs/etc.) on price, speed, accuracy, customer support, responsiveness.

Obviously it has evolved since into a huge organization and many other things have changed.

Re: If your customers don't talk, NPS is a vanity metric

#47

Post by [deleted] »

tinco wrote: post #32
superjan wrote: post #16

In stead of asking “would you reccomend”, shouldn’t you rely on “how did you learn about us?”, or “why did you buy our product”, the first one being the most reliable, as it is a neutral question about a fact.

No, the goal of tracking NPS is to find out whether the customer enjoys the product enough that they would recommend it. Whether the customer would recommend it to other potential customers is almost completely irrelevant. The article is the author realizing what NPS is meant for, not some hot take. Asking "How did you learn about us?" is a question that helps evaluate your marketing and sales pipeline. Asking "Would…

[deleted]

Re: If your customers don't talk, NPS is a vanity metric

#48

Post by mhuffman »

ggm wrote: post #6

I work in a not for profit functional monopoly space. A member body distributing resources according to policy. I can get why we do satisfaction scores, but NPS never made sense to me. Like, does the justice system do NPS for family law cases? It's not like you can get divorced in a 7-11 by the clerk, and even mediation is a process inside the model so NPS in a non competitive self-regulated monopoly.. what does it e…

>what does it even mean to recommend to others?

I will give you a real-world example. My girlfriend works for the government and they have an initiative to do outreach to people in the local community and let them know of all the funded help available to people for different situations.

However, they can't go to all 30k+ households at once and even in targeted areas (which is the way it works now) it takes a very long time to get to everyone.

The thinking about a recommendation is that perhaps they go to a home and let the person know about all the free programs available and leave a list with phone numbers. And this person might not need that information at the moment, but maybe they know someone in their family or from church or elsewhere that could use it and they let them know.

I do understand that if you help people who have lost their home due to fire it is silly to do a survey and ask them if they would or did tell someone else in case their home burns down. However, it isn't across the board silly even or not for profit cases.

Re: If your customers don't talk, NPS is a vanity metric

#49

Post by kmoser »

nulbyte wrote: post #3

I would go a step further: NPS is a garbage metric. First, you start by assuming your customers can even reasonably ascertain their likelihood to recommend. They can't; there are people who answer 10 but will never recommend, and there are people who answer 0 but already have and will again. Next, you assume your customers are idiots and don't know how an 11-point scale works by adjusting the midpoint: Instead of 5,…

NPS is the electronic version of a poorly run focus group. People will assume all sorts of things and bring in all manner of bias. You may get some data, but you have no idea how accurate it is. Even worse, you have no idea why they answered what they did, or how to fix things to make your customers happier.

Re: If your customers don't talk, NPS is a vanity metric

#50

Post by superjan »

tinco wrote: post #32
superjan wrote: post #16

In stead of asking “would you reccomend”, shouldn’t you rely on “how did you learn about us?”, or “why did you buy our product”, the first one being the most reliable, as it is a neutral question about a fact.

No, the goal of tracking NPS is to find out whether the customer enjoys the product enough that they would recommend it. Whether the customer would recommend it to other potential customers is almost completely irrelevant. The article is the author realizing what NPS is meant for, not some hot take. Asking "How did you learn about us?" is a question that helps evaluate your marketing and sales pipeline. Asking "Would…

Ok. I am clearly out of my comfort zone here. So NPS is not a proxy for organic marketing success, but for customer satisfaction. Is the article’s point then that the question is sometimes too hypothetical?

I still think that it is useful to track actual reccomendations when that is expected to happen.

Post reply on HN