Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Data will not tell you what to do

mikkeldengsoe.substack.com

Post reply on HN
Print view View original

61–70 of 75 posts

Re: Data will not tell you what to do

#61

Post by hef19898 »

nonrandomstring wrote: post #60
hef19898 wrote: post #59

Earlier quoted context omitted.

I consider myself to be rather good in my field. Which is exactly why I take every bit of data I can get before I provide my opinion or decide something. Every situation is different, facts change, so I have to evaluate my opinion each and every time (which is hownypu learn and bevome better). And the more data I have, the easier this is.

Like me, you opine and decide. Sounds like you and I are both experts who take advantage of all the tools in the box, numerical, computational and messy wetware. Sorry if you may have taken umbrage with my depiction of "dull headed data slaves". What I was referring to there are people who only use the numbers. For them there are no opinions or decisions. Only calculations. And as per TFA, it is that mentality that s…

On that, we absolutely agree! Im the end so, those who let some calculation decide for them and those who just throw solutions at the wall, are equally bad at decision making.

I know, because I made my share of bad decisions, especially early on in my career.

Re: Data will not tell you what to do

#62

Post by traceroute66 »

silverquiet wrote: post #56
traceroute66 wrote: post #48

Earlier quoted context omitted.

> Talking to real customers and helping them solve real problems is really potent. This, this and THIS again ! Example case (of many I could cite) would be Transferwise. They used to be good, but now they've denigrated into a quagmire. Could they be bothered to talk to their customers, or even just send round some box ticking surveys, they might find that out. No amount of A/B testing, data lakes or other "data scien…

I’m fairly sure there’s no legal requirement for a company to put shareholders first. I’ll sound a bit Marxist for a moment and say that we’ve just so internalized capitalist propaganda that we collectively seem to believe that now.

> I’m fairly sure there’s no legal requirement for a company to put shareholders first.

You're likely correct that there is no explicit legal requirement.

However (as I understand it), it stems from the implied requirement that derives from the fact that a company's directors have a fiduciary duty to act in good faith in the interests of the company.

People who agree with the implied requirement argue that "in the interests of the company" equates to "for the benefit of its members". And so you then ask yourself who are "its members" and that's where you end up at "its shareholders".

I believe in the jargon, this is referred to as "the common law approach of shareholder primacy".

Going back to the "legal requirement" front, there is, for example s172(1) of the Companies Act 2006[1], which starts by saying:

     "A director of a company must act in the way he considers, in good faith, would be most likely to promote the success of the company for the benefit of its members as a whole, and in doing so have regard (amongst other matters) to—"
So "must" is in the context of "benefit of its members as a whole", and a director is "only" required to "have regard" for other stakeholders that the legislation lists in (a)–(f). Its a bit of a word-salad, but effectively appears to re-enforce shareholder primacy.

[1]https://www.legislation.gov.uk/ukpga/2006/46/section/172

Re: Data will not tell you what to do

#63

Post by scott_w »

I've worked in situations where A/B testing is heavily used (and useful) and in situations where it was completely useless. Both in the same company but at separate points in the flow.

Where you have enough users (maybe you're a major online retailer), A/B testing should be a vital part of your toolkit. Not the only tool, but you definitely need to test every change you make. If you can gather enough data within 24 hours, why wouldn't you test your change?

That being said, A/B testing isn't the be-all and end-all. It just gives you some information to make a decision. You still need to know your customer, speak to them, survey, observe, etc. You might even pick a "losing" variation with the aim being to reach a more optimal business outcome. Data doesn't give you the right to abdicate your responsibility to make good decisions.

There are cases where A/B testing can't help at all. A great example is in low-volume but critical flows (think SaaS conversion funnels). For these, you need to rely on the other skills you have at your disposal.

Re: Data will not tell you what to do

#64

Post by nonrandomstring »

hef19898 wrote: post #61
nonrandomstring wrote: post #60

Earlier quoted context omitted.

Like me, you opine and decide. Sounds like you and I are both experts who take advantage of all the tools in the box, numerical, computational and messy wetware. Sorry if you may have taken umbrage with my depiction of "dull headed data slaves". What I was referring to there are people who only use the numbers. For them there are no opinions or decisions. Only calculations. And as per TFA, it is that mentality that s…

On that, we absolutely agree! Im the end so, those who let some calculation decide for them and those who just throw solutions at the wall, are equally bad at decision making. I know, because I made my share of bad decisions, especially early on in my career.

Yes me too. All my royal screw-ups came of hubris and imbalance between measurable facts and feelings I ignored. Respex.

Re: Data will not tell you what to do

#65

Post by Retric »

traceroute66 wrote: post #62
silverquiet wrote: post #56

Earlier quoted context omitted.

I’m fairly sure there’s no legal requirement for a company to put shareholders first. I’ll sound a bit Marxist for a moment and say that we’ve just so internalized capitalist propaganda that we collectively seem to believe that now.

> I’m fairly sure there’s no legal requirement for a company to put shareholders first. You're likely correct that there is no explicit legal requirement. However (as I understand it), it stems from the implied requirement that derives from the fact that a company's directors have a fiduciary duty to act in good faith in the interests of the company. People who agree with the implied requirement argue that "in the in…

Firstly, financial benefits are only one aspect of possible benefits to stockholders. A company may consider reducing pollution a direct benefit to its owners as that improves their health. There’s quite a lot of freedom for things as cancer research for example benefits people beyond the financial incentives.

Anyway, the next sections makes it explicit that shareholder primacy isn’t required:

2)Where or to the extent that the purposes of the company consist of or include purposes other than the benefit of its members, subsection (1) has effect as if the reference to promoting the success of the company for the benefit of its members were to achieving those purposes.

(3)The duty imposed by this section has effect subject to any enactment or rule of law requiring directors, in certain circumstances, to consider or act in the interests of creditors of the company.

Shareholder primacy instead stems from shareholders being able to fire management.

Re: Data will not tell you what to do

#66

Post by epgui »

wodenokoto wrote: post #6
fuidani wrote: post #5

> Intuition is underrated > Spend time where your customers are and make your own conclusions. This is a great article, very well-written, and I enjoyed reading it. However, could intuition and spending time with customers be considered another way of collecting data points to inform data-driven decisions?"

Generally when talking about data driven you are talking about quantitative data and not qualitative data.

A pet peeve of mine is when people assume qualitative data is less rigorous. It's really just as rigorous and just as valid, and can sometimes require even more sophisticated analysis.

Anyone can plug numbers into a formula (there are so few barriers to doing that, that most people probably do it kind of wrong and get approximately directionally-correct results anyway), but handling qualitative data requires really knowing what you're doing from first principles.

Re: Data will not tell you what to do

#67

Post by hef19898 »

nonrandomstring wrote: post #64
hef19898 wrote: post #61

Earlier quoted context omitted.

On that, we absolutely agree! Im the end so, those who let some calculation decide for them and those who just throw solutions at the wall, are equally bad at decision making. I know, because I made my share of bad decisions, especially early on in my career.

Yes me too. All my royal screw-ups came of hubris and imbalance between measurable facts and feelings I ignored. Respex.

I usually added a fair share of not listening to advice from people I didn't want to listen to for various reasons. Now I try my best to seperate the message from the messeneger as much as I humanly can.

Part of the learning experience, I guess?

Re: Data will not tell you what to do

#68

Post by traceroute66 »

Retric wrote: post #65
traceroute66 wrote: post #62

Earlier quoted context omitted.

> I’m fairly sure there’s no legal requirement for a company to put shareholders first. You're likely correct that there is no explicit legal requirement. However (as I understand it), it stems from the implied requirement that derives from the fact that a company's directors have a fiduciary duty to act in good faith in the interests of the company. People who agree with the implied requirement argue that "in the in…

Firstly, financial benefits are only one aspect of possible benefits to stockholders. A company may consider reducing pollution a direct benefit to its owners as that improves their health. There’s quite a lot of freedom for things as cancer research for example benefits people beyond the financial incentives. Anyway, the next sections makes it explicit that shareholder primacy isn’t required: 2)Where or to the exten…

Its clearly quite a technical topic.

But what I would say is, in relation to (2) that you highlight, that "purposes of a company" means that defined in its Articles of Association as created when it the company was formed (and as later amended if that is the case).

So I would argue (2) doesn't apply to the majority of companies, many of whom are likely operating off template Articles without expanded purpose definitions.

In relation to (3), interests of creditors, this was brought before the Supreme Court in recent history[1]. My reading of the summary of the judgement would suggest there is a relatively narrow window for being mandated vs "have regard", in particular:

"All members of the Court agree that AWA’s directors were not at the relevant time under a duty to consider, or to act in accordance with, the interests of creditors"

[1] https://www.supremecourt.uk/press-summary/uksc-2019-0046.htm...

Re: Data will not tell you what to do

#69

Post by Eridrus »

from-nibly wrote: post #7

I've never worked at a company where we were anywhere close to being able to A/B testing on customers. We just always shot from the hip. I think if you are doing that kind of testing may e you've run out of helpful ideas? Talking to real customers and helping them solve real problems is really potent. And you can get more than just the color of a button. You can get the direction your company needs to go for months.…

I think the discourse is converging on A/B testing being a big company thing, where I think it is very useful.

Not because it finds new knowledge, but because it keeps your product teams honest.

It's really easy to delude yourself and others about your project when your promotion is on the line, and A/B tests let you actually evaluate whether the change helped or not.

At small companies, you're not trying to find 2% effect sizes, anything that small is already a failure, so you don't need statistics to tell you what worked.

Re: Data will not tell you what to do

#70

Post by Arubis »

Data _can_ tell you what to do, but that doesn't mean your data selection and gathering was precise, right, and aligned to your actual best interests.

Data-informed decisionmaking is great. Data-driven decisionmaking, not so much. You still need to trust your gut.

Post reply on HN