Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Removing stuff is never obvious yet often better

gkogan.co

Post reply on HN
Print view View original

71–80 of 196 posts

Re: Removing stuff is never obvious yet often better

#71

Post by lelanthran »

shubhamjain wrote: post #60
CookieCrisp wrote: post #56

Earlier quoted context omitted.

I feel like that example is missing some context - if signups did increase then their experiment was successful - we aren’t here to make pretty pages, we’re here to make money.

The problem is that it's easy to prove that signups are increasing, and lot harder to prove that there was a measurable increase in number of paying users. Most A/B tests focus on the former, very few on the latter. We had a free plan, and most users who signed up never made a single API request. So, assuming that the increase in signups is driving more business is just foolhardy.

> We had a free plan, and most users who signed up never made a single API request.

That doesn't sound like a signup problem; what was the goal behind the free plan? Drive more paying users? Raise company profile? Lock in more users?

Re: Removing stuff is never obvious yet often better

#72

Post by kqr »

refibrillator wrote: post #53

Upvoted to spread the immense wisdom in this post. But golly I must say the line can get blurry quickly. > Would anything of value be lost if this or that chunk of it was removed? In early stage projects I’ve seen this mentality backfire occasionally because it’s tough to estimate future value, especially for code and data. For example, one time in a greenfield project I created the initial SQL schema which had some…

I think when you have a mindset of removing the useless (which your columns were at at the time) you have to be prepared to sometimes add things back in. Yes, it is painful, but it is not a signal that the process didn't work. You should expect to sometimes have to add things back in.

We cannot perfectly predict the future, so when removing things there will always be false positives. The only way to get a false positive rate of zero is to never remove anything at all.

The real question is what level of false positives we find acceptable. If you can only cite this one case where it was painful, then I'd say that's evidence your colleagues approach worked very well!

(I think Elon Musk recommends a deletion false positive rate of 10 % as appropriate in general, but it will vary with industry.)

Re: Removing stuff is never obvious yet often better

#73

Post by due-rr »

refibrillator wrote: post #53

Upvoted to spread the immense wisdom in this post. But golly I must say the line can get blurry quickly. > Would anything of value be lost if this or that chunk of it was removed? In early stage projects I’ve seen this mentality backfire occasionally because it’s tough to estimate future value, especially for code and data. For example, one time in a greenfield project I created the initial SQL schema which had some…

The big difference here is something the user sees versus the developers as in your example. For users I think the answer is almost always, less is better. For the developers also, but there’s a bit more leeway.

Re: Removing stuff is never obvious yet often better

#74

Post by grujicd »

refibrillator wrote: post #53

Upvoted to spread the immense wisdom in this post. But golly I must say the line can get blurry quickly. > Would anything of value be lost if this or that chunk of it was removed? In early stage projects I’ve seen this mentality backfire occasionally because it’s tough to estimate future value, especially for code and data. For example, one time in a greenfield project I created the initial SQL schema which had some…

Main issue with adding something you might need in the future is that people leave, people forget, and then, one year later, there's some metadata column but no one remembers whether it was already used for something. Can we use it? Should we delete it? Someone remembers Knight Capital and spectacular failure when old field was reused. So it's always safer to keep existing field and then you end up with metadata and metadata_1. Next year no one remembers why there are two metadata fields and is very confused on which one should be used.

Re: Removing stuff is never obvious yet often better

#75

Post by mrweasel »

refibrillator wrote: post #53

Upvoted to spread the immense wisdom in this post. But golly I must say the line can get blurry quickly. > Would anything of value be lost if this or that chunk of it was removed? In early stage projects I’ve seen this mentality backfire occasionally because it’s tough to estimate future value, especially for code and data. For example, one time in a greenfield project I created the initial SQL schema which had some…

In this case you where right, and the removal of the additional fields turned out to be a mistake. More often though I see this going the other way, you end up carrying around tables and fields that aren't needed and just muddles the picture. Even worse is "Well we added this field, but we're not using it, so let just stuff something else in there". It can get really ugly really fast.

The article is a little different, because "YES, they do need the calculator, people should know what your service costs up front". If they want to do away with the calculator, then they should restructure their pricing plans, so that it will no longer be required. That just not an IT problem.

Re: Removing stuff is never obvious yet often better

#76

Post by infogulch »

refibrillator wrote: post #53

Upvoted to spread the immense wisdom in this post. But golly I must say the line can get blurry quickly. > Would anything of value be lost if this or that chunk of it was removed? In early stage projects I’ve seen this mentality backfire occasionally because it’s tough to estimate future value, especially for code and data. For example, one time in a greenfield project I created the initial SQL schema which had some…

I'm sympathetic to your situation, but it's possible that the senior was still right to remove it at the time, even if you were eventually right that the product would need it in the end.

If I recall correctly they have a measure at SpaceX that captures this idea: The ratio of features added back a second time to total removed features. If every removed feature was added back, a 100% 'feature recidivism' (if you grant some wordsmithing liberty), then obviously you're cutting features too often. 70% is too much, even 30%. But critically 0% feature recidivism is bad too because it means that you're not trying hard enough to remove unneeded features and you will accumulate bloat as a result. I guess you'd want this ratio to run higher early in a product's lifecycle and eventually asymptote down to a low non-zero percentage as the product matures.

From this perspective the exact set of features required to make the best product are an unknown in the present, so it's fine to take a stochastic approach to removing features to make sure you cut unneeded ones. And if you need to add them back that's fine, but that shouldn't cast doubt on the decision to remove it in the first place unless it's happening too often.

Alternatively you could spend 6 months in meetings agonizing over hypotheticals and endlessly quibbling over proxies for unarticulated priors instead of just trying both in the real world and seeing what happens...

Re: Removing stuff is never obvious yet often better

#77

Post by duped »

To be a sociopath for a second:

Up-front pricing is only good for the buyer and never good for the seller. And while being an advocate for the user is a good thing in general, it's not a good thing if the result is less revenue. And if you want to present a pricing estimate your incentive is always to lowball and underestimate.

Businesses that are successful tend to exploit information asymmetry. This is frustrating as a consumer but if you can guarantee that the user doesn't know their final cost you can always sell them a sweeter picture than reality and play off the cost of switching to a competitor, if one exists.

Total aside but this is why housing prices are insane, at the most micro level no buyer has any idea over the final cost until the last possible moment at which point the cost of undoing the transaction probably outweighs the risk of continuing - psychologically, at least.

(I may or may not have been the victim of this recently and thought about it from the seller's side)

Re: Removing stuff is never obvious yet often better

#78

Post by shahzaibmushtaq »

We, as humans, always carry a few types of things with us on a journey.

Things we like, things we don't like, things that are totally useful, things that are somewhat useful etc.

There comes a point where we need to let go of the thing(s) we like - I called them the precious payload where we are most reluctant - and in this case the 'calculator' was the precious payload so many people in the company were unwilling to remove this feature except for one person.

In business, adding a completely new feature or subtracting an age-old feature is extremely difficult but oftentimes, this is where growth comes from.

Re: Removing stuff is never obvious yet often better

#79

Post by jessriedel »

mquander wrote: post #22

I don't know if this calculator was good or bad, but the rationale sounds superficially ridiculous. > Visitors who didn't see the calculator were 16% more likely to sign up and 90% more likely to contact us than those who saw it. There was no increase in support tickets about pricing, which suggests users are overall less confused and happier. Of course if you hide the fact that your product might cost a lot of money…

Relatedly, there seemed to be no acknowledgement of the possibility of dark incentives: many businesses have found they can increase sales by removing pricing details so that prospective customers get deeper into the funnel and end up buying because of sunk time costs even though they would have preferred a competitor. Example: car dealerships make it a nightmare to get pricing information online, and instead cajole you to email them or come in in person. In other words, a calculator makes it easier to comparison shop, which many businesses don't like.

I have no idea if that's a conscious or unconscious motivation for this business, but even if its not conscious it needs to be considered.

Re: Removing stuff is never obvious yet often better

#80

Post by InDubioProRubio »

grujicd wrote: post #74
refibrillator wrote: post #53

Upvoted to spread the immense wisdom in this post. But golly I must say the line can get blurry quickly. > Would anything of value be lost if this or that chunk of it was removed? In early stage projects I’ve seen this mentality backfire occasionally because it’s tough to estimate future value, especially for code and data. For example, one time in a greenfield project I created the initial SQL schema which had some…

Main issue with adding something you might need in the future is that people leave, people forget, and then, one year later, there's some metadata column but no one remembers whether it was already used for something. Can we use it? Should we delete it? Someone remembers Knight Capital and spectacular failure when old field was reused. So it's always safer to keep existing field and then you end up with metadata and…

So, every db-column gets a "in_use_boolean" assigned? It gets reset every year, reset on first use query and auto-purged after a year and a day. Self-pruning database..
Post reply on HN