Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Be good-argument-driven, not data-driven

twitchard.github.io

Post reply on HN
Print view View original

151–160 of 168 posts

Re: Be good-argument-driven, not data-driven

#151

Post by tdehnel »

jasode wrote: post #94
tdehnel wrote: post #55

Earlier quoted context omitted.

A simple answer to this is that good explanations are hard to vary. More here: https://www.lesswrong.com/posts/jcTsbaQ8hNc7qxwaQ/explanatio...

>A simple answer to this is that good explanations are hard to vary. But the "hard to vary" explanations were built up from observing data of smashing particles. E.g. from your link: - Frank Wilczek describes hard-to-vary-ness as follows "A theory begins to be perfect if any change makes it worse." He explains further using the Standard Model as an example of a hard-to-vary explanation: Too many gluons! But each of t…

Duh, science is the interplay between observations and explanations. But that doesn’t change that what makes an explanation good or bad is whether it’s hard to vary.

I’ll use a David Deutsch example: let’s say a theory that eating 1kg of grass cures the common cold.

You could do an experiment and find it does not. But you could easily vary the theory and say actually it’s 1.1kg and so on.

But you wouldn’t actually need to do the experiment because there is no good, invariable explanation as to _why_ eating the grass cures a cold.

In that case, you wouldn’t need any data or observations at all. You could simply ask why eating exactly 1kg of grass cures the cold. What is the mechanism of action?

In this way you can see that empiricism is not sufficient in any case to give evidence to a theory. We need only a good explanation to judge whether a theory is worth considering to be true. From there, we can do further experiments/observations to rule it out. But never to prove it true.

Re: Be good-argument-driven, not data-driven

#152

Post by tdehnel »

oxfordmale wrote: post #93
tdehnel wrote: post #54

Earlier quoted context omitted.

> Any good-argument-driven based argument you attempt to make is almost always based on political motivating factors If this is true in the case of a specific theory, then that is not a good theory.

I was mostly referring to business decisions. For that type of decisions there are always political factors at play (building empires, career growth, dislike for another person/team) that do not necessarily align with business success. Lehman Brothers is one of those examples.

You’re actually proving my point. Those so-called successful business decisions never turn out to be true enough to work long term. Like all knowledge, they are eventually shown to be false.

Re: Be good-argument-driven, not data-driven

#153

Post by oxfordmale »

tdehnel wrote: post #152
oxfordmale wrote: post #93

Earlier quoted context omitted.

I was mostly referring to business decisions. For that type of decisions there are always political factors at play (building empires, career growth, dislike for another person/team) that do not necessarily align with business success. Lehman Brothers is one of those examples.

You’re actually proving my point. Those so-called successful business decisions never turn out to be true enough to work long term. Like all knowledge, they are eventually shown to be false.

Data driven companies, like Amazon, tend to do better, on average than other companies. However, this doesn't mean they are immune to mistakes. At some point in time Amazon wil fall in decline and disappear, however, that doesn't imply that being data driven was a bad decision.

Re: Be good-argument-driven, not data-driven

#154

Post by jason-phillips »

zmgsabst wrote: post #36
jason-phillips wrote: post #33

This reminds me of the Principal Chalmers meme. In this case, first pondering whether he is wrong, only to conclude that it's the data that's wrong. I know that's not what the article says per se, but it's only one slightly abstracted reinterpretation removed, as OP's title demonstrates.

Minor nit: Principal Skinner; Chalmers was the superintendent. https://www.knowyourmeme.com/memes/am-i-so-out-of-touch

Doh!

Re: Be good-argument-driven, not data-driven

#155

Post by guerrilla »

crabmusket wrote: post #149
guerrilla wrote: post #24

Earlier quoted context omitted.

What do you mean that they went too far? James and Peirce were not describing "reality" (in this discussion anyway. [1][2]) but rather were instrumentalists and thus saw every theory as having a purpose. That's the whole point of the squirrel argument. It not just "depends on what you mean" (as per analytic and some medieval philosophy) but also depends on what you're trying to do (which in turn depends on what you w…

I think I'm out of my depth at this point actually, and maybe shouldn't have opined as readily as I did on the pragmatists. I did a little recap of where I'd encountered the pragmatists before and realised I only read Royce, who was a friend and interlocutor of James. But I don't think he could be called a member of the pragmatist school, so I shouldn't take the impressions I got from him to be representative! The im…

Oh okay. I'm not an expert either but I think many might even consider them anti-realists but I'm not sure... I have a friend who is literally an expert, so I should probably ask and figure that out :)

> Deutsch has a fair bit to say about instrumentalism in Beginning of Infinity which I will leave to the interested reader to discover.

Okay, thanks, I'll check it out!

Re: Be good-argument-driven, not data-driven

#156

Post by blueyes »

Key idea is the "data maturity" of the topic under discussion.

Where there is data, you should use it and be smart about it.

For a lot of big decisions, especially in companies doing something new, there is no good data at first. You have to reason about it based on experience and analogy.

Then, once you commit to a path, you can start gathering data to see if your hypothesis was correct. The further you go, the more you can rely on data, assuming you know how to think about it.

Discussions about being data-driven that don't take into account the "data maturity" of the situation are nonsensical.

Being "data driven" when you're considering something radically new is either delusional or a cop out.

Ignoring data when it could correct your biases is either lazy or wrong or both.

And finally, lots of people who claim to be "data driven" are not smart about data. To paraphrase Wilde, "data is rarely pure and never simple." It doesn't just reveal truths you can treat as dogma. It's ambiguous and takes a lot of work to interpret. A lot of "data driven" teams aren't doing that work.

Re: Be good-argument-driven, not data-driven

#157

Post by tlarkworthy »

btilly wrote: post #137
tlarkworthy wrote: post #82

Earlier quoted context omitted.

ycombinator startup school disagrees and says it's one of the two CRITICAL things founders must have a hand in. Of course you need to interpret it but its incredibly important and I do not think you really know what you are talking about. https://www.ycombinator.com/library/6g-how-to-talk-to-users Almost all the major fails I have seen in my career have been some derivative of not understanding your users.

That depends on how big the differences you're looking for are. When you've got an early product, there are probably things you can do that 2x as many people will like as dislike. Even a small set of customers will be good for discovering this. When you've got a mature product, you should be optimizing around the edges and need a large sample size to find those 1% wins. Likewise if you don't have scale, there are a l…

It's this data-led uncritical thinking that destroyed facebook

Re: Be good-argument-driven, not data-driven

#158

Post by leetcrew »

thiele wrote: post #97
dxbydt wrote: post #84

Earlier quoted context omitted.

frankly, there’s only a tiny handful of these mythical saas “1000 users each paying 1 mullion dollars” companies. the vast, vast majority of saas startups are serving millions of “users” - i put that in quotes because these aren’t real users or customers. they are real people checking out your product - but they aren’t users or customers. if you set up a gas station near the off ramp of some major interstate, say I-6…

This is a really good analogy (gas station customers) that I haven’t heard before. I’ve often tried to describe this ‘low intent’ group but never had a good way to make it relatable.

it gets the point across, but it raises other interesting questions.

sure, most customers at an interstate gas station will only visit once or twice, but that doesn't necessarily mean they are less important to the business than the truck drivers that fill up every day. maybe the bulk of revenue actually comes from one-time customers. this could be a case where attracting new customers is more important than retaining the current ones.

Re: Be good-argument-driven, not data-driven

#159

Post by jjk166 »

gwd wrote: post #30
quanto wrote: post #6

> A weak argument founded on poorly-interpreted data is not better than a well-reasoned argument founded on observation and theory. So a good argument is founded on...good data and good understanding of data? The article more seriously makes the mistake of begging the question: it presupposes the known classier of good and bad arguments and then goes on to say bad arguments with data is worse than good arguments. But…

> It presupposes the known classier of good and bad arguments and then goes on to say bad arguments with data is worse than good arguments. It does indeed assume that there's a way to learn bad arguments from good; and so the focus should be on learning what are good argument and what are bad. > ...What makes a good argument if not empirical data? Consider the following conversation: A: We've done some numbers, and w…

You left out hypothesis C: if you send too many firefighters they get in eachothers way and become difficult to coordinate, making them worse at putting out the blaze.

And hypothesis D: fires with the same number of firefighters cause different levels of destruction because some departments are organized to let their 10X firefighters work more efficiently.

And hypothesis E: Many arsonists become firefighters thus more firefighters increases the risk that an arsonist will be on the team

And hypothesis F: The same as hypothesis A but since some tools require more than one person there's actually a minimum threshold below which destruction skyrockets

And hypothesis G: Wealthy areas that can hire more firefighters also suffer more expensive destruction for a given blaze.

And hypothesis H: If we invest the resources we're spending on firefighters into fire prevention we can reduce total fire damage

And infinitely more hypotheses.

There will always be another argument that makes some logical sense. And unfortunately reality is under no obligation to make sense, so it's entirely possible something that sounds stupid and counterintuitive could just happen to be correct anyways.

But with data, we can test hypotheses. Vary the number of firefighters and see what happens.

Re: Be good-argument-driven, not data-driven

#160

Post by butUhmErm »

oivey wrote: post #138
butUhmErm wrote: post #81

Earlier quoted context omitted.

Excel has a history of forced format updates, breaking incompatibility. I know people who banned it because they got tired of marching to MSs upgrade beat. Python 2 to 3 upgrade aside, can’t really say the same about the language. There are a number of good arguments out there that might violate an engineers perception, which one might call a cognitive data model built through training and experience. There is no the…

Libraries introduce breaking changes, too. I’ve been bit by silent default changes in Pandas, for example. To me that’s kind of striking because I also wouldn’t consider myself a major user of the library.

No post body was provided.
Post reply on HN