Yes, data is useless without a qualitative explanation. There are simply too many possible confounding factors that you cannot eliminate without understanding what they may be.
Be good-argument-driven, not data-driven
111–120 of 168 posts
Re: Be good-argument-driven, not data-driven
#112The related problem that I see actually more often is the "you don't have big data" problem. You know, in data science, you see people spending hours writing pandas scripts that replicate a few clicks in excel for a one of analysis. You see datasets of a few gigabytes being processed with spark when SQL would be fine. You see ML techniques being thrown at questions that could be answered simply and reliably with basi…
Talking to people is not going to help you either. You end up getting a lot of noise and making sense of what you hear is difficult. When you keep probing you will get to hear stuff thats not really critical and just often made up because you ask too many questions. Classical trap of market research.
> You end up getting a lot of noise and making sense of what you hear is difficult. When you keep probing you will get to hear stuff thats not really critical and just often made up because you ask too many questions.
Yeah, but this just means qualitative data is challenging, not that it's useless. You have to be careful when asking questions that you're asking useful questions and not leading people into telling you what they think you want to hear (or going off on useless rabbit trails like what they think the product should be instead of what the problem they want the product to solve is).
Re: Be good-argument-driven, not data-driven
#113Being argument driven gives control to the organization's 'lawyers'. People can be very persuasive independent of the reality of the situation.
Re: Be good-argument-driven, not data-driven
#114This entire discussion makes a good case for why the general populace would benefit from being taught the basics of philosophy. In this case the topic of value is the often fraught relationship between empiricism and rationalism , and the impacts each have on the scientific process, research, education, and how we go about understanding the world. To operate with one with a complete absence of the other is to expose…
While this is indeed an issue that falls within the domain of philosophy, philosophy is also home to fields in which the absence of empirical evidence is regarded as an irrelevance, or at best a mere detail that can be deferred to an indefinite future. Take, for example, the resurgence of enthusiasm for panpsychism, and the enduring appeal of armchair metaphysics. I am doubtful that academic philosophy has much enthu…
Exploring ideas like panpsychism doesn't mean you're committing to them being true. We can't know everything, and we can't always link new ideas deductively to things we are certain about, but we can notice the inadequacy of current explanations, say "suppose this explanation is true" and proceed from there. Every good philosopher knows that they're doing that. And the fact that people defend their position and attack opposing views is just part of the adversarial process for testing ideas. Yeah, of course ego and pride and hubris happen to many philosophers, and the academic profession is frankly in a bad state, but that doesn't mean the fundamental approach is bad.
Re: Be good-argument-driven, not data-driven
#115Be data-driven, and question the provenance of your data all the time. Otherwise you will end up like economics, a field with prettier models and more mathematics than almost every engineering field, and yet gets every major prediction wrong.
Re: Be good-argument-driven, not data-driven
#116Earlier quoted context omitted.
While this is indeed an issue that falls within the domain of philosophy, philosophy is also home to fields in which the absence of empirical evidence is regarded as an irrelevance, or at best a mere detail that can be deferred to an indefinite future. Take, for example, the resurgence of enthusiasm for panpsychism, and the enduring appeal of armchair metaphysics. I am doubtful that academic philosophy has much enthu…
Exploring ideas like panpsychism doesn't mean you're committing to them being true. We can't know everything, and we can't always link new ideas deductively to things we are certain about, but we can notice the inadequacy of current explanations, say "suppose this explanation is true" and proceed from there. Every good philosopher knows that they're doing that. And the fact that people defend their position and attac…
That is a fair point in general, but in the specific case of panpsychism, at least one of its most active proponents (Goff) combines an insistance that it is the most plausible explanation of the mind with an apparent lack of interest in saying anything empirically verifiable about what it actually means.
Whether in physics or metaphysics, one can only go so far without facts. Even the mundane world of that which actually is has repeatedly turned out to be stranger than was imagined possible.
Re: Be good-argument-driven, not data-driven
#117While I agree completely with the premise of this article, on the other hand I'm weighing the relatively robust findings by Meehl et al. They find, time and time again, in all sorts of fields, that extremely parsimonious models like equal-weighted linear regression of one or two predictors outperform expert judgment[1]. One would think this is cognitively dissonant enough, but it gets worse: This article, with the th…
> They find, time and time again, in all sorts of fields, that extremely parsimonious models like equal-weighted linear regression of one or two predictors outperform expert judgment.
I came across this in Thinking Fast and Slow. Kahneman was a big fan of Meehl and restates the point:
The important conclusion from this research is that an algorithm that is constructed on the back of an envelope is often good enough to compete with an optimally weighted formula, and certainly good enough to outdo expert judgment.
https://www.goodreads.com/quotes/9574537-the-important-concl...
I too agree with the premise of this article. On this topic of expert judgment vs data, however, I found the counterpoint in this HN comment thought-provoking enough to bookmark and refer back to now and again:
I started at MS during Vista and I've been involved (sometimes tangentially) with Windows ever since. This is all my opinion, but It's been very interesting seeing the decision making process change over time.
If I had to summarize the change, I'd say that it's evolved from an expertise-based system to a data based system. The reason why eight people were present at every planning meeting is because their expert opinion was the primary tool used in decision making. In addition to poor decisions, this had two very negative outcomes:
1) reputation was fiercely fought for. Individuals feared that if they were ever incorrect, the damage to their reputation would limit their ability to impact future decisions and eventually lead to career death. Whether this actually happened or not is irrelevant; the fear itself caused overt caution and consensus seeking.
2) In the absence of data, an eloquent negotiator is often able to obtain their desired outcome, no matter how sub-optimal that outcome might be.
https://news.ycombinator.com/item?id=15174737#15176957
Even more provocative, it ends up being a (qualified, as I read it) defense of telemetry.
Re: Be good-argument-driven, not data-driven
#118While I agree completely with the premise of this article, on the other hand I'm weighing the relatively robust findings by Meehl et al. They find, time and time again, in all sorts of fields, that extremely parsimonious models like equal-weighted linear regression of one or two predictors outperform expert judgment[1]. One would think this is cognitively dissonant enough, but it gets worse: This article, with the th…
> This article, with the thesis that good arguments are more important than data, is based on, well, a good argument – not much data.
I'm not sure what you're claiming. All intellectual demonstration is a matter of rational argument. That's what proofs are: arguments. Data is not self-explanatory or demonstration. "Data" can only support arguments by first being collected, something motivated by argument, and then interpreted so that it can enter into argument as a body of propositions.
> On the other hand, the work by Meehl et al. claiming pretty much the opposite, is based on, well, a lot of data, and maybe not much intuitive reasoning.
I don't understand. Argument is logical demonstration. The strongest form is the deductive argument. If you don't have a logical argument, then you haven't got a demonstration.
> I don't know what to believe. Fortunately, as I've grown older, I've become more comfortable with holding completely dissonant opinions in my head at the same time.
Depending on what you mean, this could be good or bad. Inconsistency is not a virtue, and if there is an inconsistency between two of your beliefs, then it means you've got work to do (or at least you'll need to admit you don't know what the truth is). This requires humility, the frank acknowledgment that you're faced with an aporia that you don't know (at least not yet) how to address. It also requires patience if you are to tolerate your ignorance instead of jumping to some ersatz explanation.
Re: Be good-argument-driven, not data-driven
#119I’ll probably be buried in all these comments, but my position is that data is only as good as how it is collected. Sloppy data collection gives rise to sloppy conclusion through unknown biases.
The key is to understand the ‘data generation process’ so you can identify biases. My experience suggests that doing so side-step some common pitfalls.
I recommend reach out for ‘The Book Of Why’ by Judea Pearl. He includes many real life examples that’s surprisingly applicable to modern data science.
Re: Be good-argument-driven, not data-driven
#120Earlier quoted context omitted.
Why? Inadequately “technical”?
I think there are lots of reasons why. One possible reason: no one whose job it is to write Python scripts was ever promoted for making an Excel spreadsheet when that is the simpler and more practical approach. And no manager of people who write Python scripts is going to be able to use that Excel spreadsheet to sell "I need more responsibility and head count." People tend to follow incentives, rather than focusing o…
> People tend to follow incentives, rather than focusing on making wise decisions.
This is the key issue. Solving it isn't easy -- it requires people who are wise, and wisdom is a scarce commodity.