Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Removing stuff is never obvious yet often better

gkogan.co

Post reply on HN
Print view View original

111–120 of 196 posts

Re: Removing stuff is never obvious yet often better

#111

Post by autonomousErwin »

jessriedel wrote: post #79
mquander wrote: post #22

I don't know if this calculator was good or bad, but the rationale sounds superficially ridiculous. > Visitors who didn't see the calculator were 16% more likely to sign up and 90% more likely to contact us than those who saw it. There was no increase in support tickets about pricing, which suggests users are overall less confused and happier. Of course if you hide the fact that your product might cost a lot of money…

Relatedly, there seemed to be no acknowledgement of the possibility of dark incentives: many businesses have found they can increase sales by removing pricing details so that prospective customers get deeper into the funnel and end up buying because of sunk time costs even though they would have preferred a competitor. Example: car dealerships make it a nightmare to get pricing information online, and instead cajole…

Would you consider doubling your prices so users perceive your product as having higher value a dark pattern?

Re: Removing stuff is never obvious yet often better

#112

Post by richardw »

mquander wrote: post #22

I don't know if this calculator was good or bad, but the rationale sounds superficially ridiculous. > Visitors who didn't see the calculator were 16% more likely to sign up and 90% more likely to contact us than those who saw it. There was no increase in support tickets about pricing, which suggests users are overall less confused and happier. Of course if you hide the fact that your product might cost a lot of money…

I designed an internal system that optimises for long term outcomes. We do nothing based on whether you click “upgrade”. We look at the net change over time, including impact to engagement and calls to support months later and whether you leave 6 months after upgrading. Most of the nudges are purely for the customer’s benefit because it’ll improve lifetime value.

Re: Removing stuff is never obvious yet often better

#113

Post by bruce511 »

strken wrote: post #69
bruce511 wrote: post #54

Earlier quoted context omitted.

>> Of course, the pricing page became plain ugly, but that didn't matter, because "Signups are increasing!!" I'm not sure I'm following you here, so perhaps you'd care to elaborate? The GP critique was that it was perhaps just creating a problem elsewhere later on. I'm not seeing the similarity to your case where the change is cosmetic not functional. The issue of whitespace (padding) is subjective (see the conversat…

I think one problem is that a better design would move the button above the fold without ruining the spacing, and therefore achieve a better result with even higher lift, but someone focused on just the numbers wouldn't understand this. The fact that the A/B test has a big green number next to it doesn't mean you should stop iterating after one improvement.

A/B testing has something to say about (ideally) a single choice.

It has nothing to say about optimal solutions. Nothing about A/B testing suggests that you have reached an optimal point, or that you should lock in what you have as being the "best".

Now that the button position (or tighter layout) has been noted to have a material effect, more tests can be run to determine any more improvements.

Re: Removing stuff is never obvious yet often better

#114

Post by bruce511 »

BoredPositron wrote: post #110
carlmr wrote: post #105

Earlier quoted context omitted.

It still seems like a valid use-case for AB testing. Ideally, you should maybe redo the design, something you could AB test if it helps. My guess is yes, because consistency in design usually makes people assume better quality.

A/B tests suck because you are testing against two cases which are probably not the best case. If you take your learnings of the A/B test and iterate your design that's a viable strategy but proposing a shit design and insisting on deploying is wrong.

That's like saying that comparing a veggie burger to a normal burger sucks because neither are ice cream.

A/B tests, by definition, test between A and B. It is very likely that neither is the best option.

But how will you find the best option if you don't measure options against each other.

Re: Removing stuff is never obvious yet often better

#115

Post by otherme123 »

mquander wrote: post #22

I don't know if this calculator was good or bad, but the rationale sounds superficially ridiculous. > Visitors who didn't see the calculator were 16% more likely to sign up and 90% more likely to contact us than those who saw it. There was no increase in support tickets about pricing, which suggests users are overall less confused and happier. Of course if you hide the fact that your product might cost a lot of money…

> Of course if you hide the fact that your product might cost a lot of money from your users, more of them will sign up

The problem with their calculator was that the users introduced slightly wrong data, or misunderstand what means some metric, and suddenly a 1000x the real price was shown. Their dilemma was "how to fix those cases", and the solution was "get rid of the messy calculator".

But they are not hidding a 1000x cost, they are avoiding losing users that get a wrong 1000x quote.

Re: Removing stuff is never obvious yet often better

#116

Post by BoredPositron »

bruce511 wrote: post #114
BoredPositron wrote: post #110

Earlier quoted context omitted.

A/B tests suck because you are testing against two cases which are probably not the best case. If you take your learnings of the A/B test and iterate your design that's a viable strategy but proposing a shit design and insisting on deploying is wrong.

That's like saying that comparing a veggie burger to a normal burger sucks because neither are ice cream. A/B tests, by definition, test between A and B. It is very likely that neither is the best option. But how will you find the best option if you don't measure options against each other.

The problem is that in 90% of companies the decision is between A/B and not the iterations of them.

Re: Removing stuff is never obvious yet often better

#117

Post by raverbashing »

mquander wrote: post #22

I don't know if this calculator was good or bad, but the rationale sounds superficially ridiculous. > Visitors who didn't see the calculator were 16% more likely to sign up and 90% more likely to contact us than those who saw it. There was no increase in support tickets about pricing, which suggests users are overall less confused and happier. Of course if you hide the fact that your product might cost a lot of money…

> so we dug into it and realized the calculator was far more confusing and sensitive than we thought. One slight misinterpretation and wrong input and you'd get an estimate that's overstated by as much as 1,000x.

They should have looked into this to see how to make it more obvious or more reflective of "regular use case"

Their sliders there are not too detailed. For example, what are namespaces, how many would a typical use need? Is 100 too much or too little? And if this is one of the variables that is too sensitive they would need to represent this in a different way

Re: Removing stuff is never obvious yet often better

#118

Post by xorcist »

otherme123 wrote: post #115
mquander wrote: post #22

I don't know if this calculator was good or bad, but the rationale sounds superficially ridiculous. > Visitors who didn't see the calculator were 16% more likely to sign up and 90% more likely to contact us than those who saw it. There was no increase in support tickets about pricing, which suggests users are overall less confused and happier. Of course if you hide the fact that your product might cost a lot of money…

> Of course if you hide the fact that your product might cost a lot of money from your users, more of them will sign up The problem with their calculator was that the users introduced slightly wrong data, or misunderstand what means some metric, and suddenly a 1000x the real price was shown. Their dilemma was "how to fix those cases", and the solution was "get rid of the messy calculator". But they are not hidding a…

Let's not take a PR piece completely at face value.

There's probably a bit of both, at the very least.

Re: Removing stuff is never obvious yet often better

#120

Post by sethammons »

richardw wrote: post #112
mquander wrote: post #22

I don't know if this calculator was good or bad, but the rationale sounds superficially ridiculous. > Visitors who didn't see the calculator were 16% more likely to sign up and 90% more likely to contact us than those who saw it. There was no increase in support tickets about pricing, which suggests users are overall less confused and happier. Of course if you hide the fact that your product might cost a lot of money…

I designed an internal system that optimises for long term outcomes. We do nothing based on whether you click “upgrade”. We look at the net change over time, including impact to engagement and calls to support months later and whether you leave 6 months after upgrading. Most of the nudges are purely for the customer’s benefit because it’ll improve lifetime value.

You could only be measuring in aggregate, no? Overall signal could be positive but one element happens to be negative while another is overly positive.
Post reply on HN