Be good-argument-driven, not data-driven
twitchard.github.io
Be good-argument-driven, not data-driven
1–10 of 168 posts
Re: Be good-argument-driven, not data-driven
#2[deleted]
Re: Be good-argument-driven, not data-driven
#3> I originally claimed that data-driven culture leads bad arguments involving data to be favored over good arguments that don’t
This is symptomatic of the deeper problem of thinking in terms of bumper stickers and slogans, instead of thinking from first principles. When it afflicts educated people, usually you hear slogans like "an anecdote is not data", or "that's the slippery slope fallacy". Instead of grappling with noisy reality, they have sharp cognitive categories with firm boundaries between concepts, then they try to squeeze things into these categories in order to make cognition easier because the relations between the categories are already understood. This gives them the illusion of rigorous and clear thought.
Re: Be good-argument-driven, not data-driven
#4While I agree completely with the premise of this article, on the other hand I'm weighing the relatively robust findings by Meehl et al. They find, time and time again, in all sorts of fields, that extremely parsimonious models like equal-weighted linear regression of one or two predictors outperform expert judgment[1].
One would think this is cognitively dissonant enough, but it gets worse:
This article, with the thesis that good arguments are more important than data, is based on, well, a good argument – not much data. On the other hand, the work by Meehl et al. claiming pretty much the opposite, is based on, well, a lot of data, and maybe not much intuitive reasoning. (There's some, yes, but the main thrust of why I believe it is that variants of the experiment have been replicated reliably.)
I don't know what to believe. Fortunately, as I've grown older, I've become more comfortable with holding completely dissonant opinions in my head at the same time.
----
Edit a few minutes later: This actually prompted me to refresh on the subject. It might be the case that Meehl is actually making the same argument as this article, only it gets distorted when repeated. Some things are reliably measurable; for those things be data-driven. Other things not so much, then use your expertise.
----
[1]: Here's just one relatively early example: http://apsychoserver.psych.arizona.edu/JJBAReprints/PSYC621/...
Re: Be good-argument-driven, not data-driven
#5I often experience the inverse: people come up with hypotheses and theories that should see expressions in observable data - but no-one bothers to look and instead everyone argues around logical constructs etc.
Re: Be good-argument-driven, not data-driven
#6> A weak argument founded on poorly-interpreted data is not better than a well-reasoned argument founded on observation and theory.
So a good argument is founded on...good data and good understanding of data?
The article more seriously makes the mistake of begging the question: it presupposes the known classier of good and bad arguments and then goes on to say bad arguments with data is worse than good arguments. But how do you know good arguments from bad arguments in the first place? What makes a good argument if not empirical data?
Re: Be good-argument-driven, not data-driven
#7While I agree completely with the premise of this article, on the other hand I'm weighing the relatively robust findings by Meehl et al. They find, time and time again, in all sorts of fields, that extremely parsimonious models like equal-weighted linear regression of one or two predictors outperform expert judgment[1]. One would think this is cognitively dissonant enough, but it gets worse: This article, with the th…
I feel like the author is leaning into comfort, intuitiveness. You bring up a fantastic point. Often we find data reveals things very unintuitive to human experience. We should always try to make Good Arguments - but without data they aren't always honest beyond feelings.
Re: Be good-argument-driven, not data-driven
#8One of the big reasons why data driven approaches are so seductive is, it's very difficult in the moment to distinguish between a good argument and a well crafted rationalization.
Re: Be good-argument-driven, not data-driven
#9This reminds me a lot of the discussion of the scientific method by Karl Popper, and David Deutsch who was very influenced by Popper. "Being data-driven" sounds very empirical. Just look at the data, and see what you find in it.
But you can't just let the data "speak for itself" without an explanation or a theory that interprets the data. Popper in Conjectures and Refutations:
> Observation is always selective. It needs a chosen object, a definite task, an interest, a point of view, a problem. And its description presupposes a descriptive language ... which in its turn presupposes interests, points of view, and problems.
Deutsch, in The Beginning of Infinity, emphasizes the importance of conjecture, and the role of observation as refuting or criticising those conjectures:
> Where does [knowledge] come from? Empiricism said that we derive it from sensory experience. This is false. The real source of our theories is conjecture, and the real source of our knowledge is conjecture alternating with criticism. We create theories by rearranging, combining, altering and adding to existing ideas with the intention of improving upon them. The role of experiment and observation is to choose between existing theories, not to be the source of new ones. We interpret experiences through explanatory theories, but true explanations are not obvious.
To bring this back to the subject of the article, I might suggest that it's possible to be "data driven" without a sound explanation or theory that the data is either interpreted through, or used to criticise. Or maybe such theories do exist, but are left implicit.
Re: Be good-argument-driven, not data-driven
#10This reminds me a lot of the discussion of the scientific method by Karl Popper, and David Deutsch who was very influenced by Popper. "Being data-driven" sounds very empirical . Just look at the data, and see what you find in it. But you can't just let the data "speak for itself" without an explanation or a theory that interprets the data. Popper in Conjectures and Refutations : > Observation is always selective. It…
> But you can't just let the data "speak for itself" without an explanation or a theory that interprets the data.
If you look at the heart attack data, and you ignore smoking you end up inventing the mythical Type A personality — but it was data driven.
https://en.m.wikipedia.org/wiki/Type_A_and_Type_B_personalit...