Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Be good-argument-driven, not data-driven

twitchard.github.io

Post reply on HN
Print view View original

61–70 of 168 posts

Re: Be good-argument-driven, not data-driven

#61

Post by Cthulhu_ »

ekianjo wrote: post #42
shubb wrote: post #35

The related problem that I see actually more often is the "you don't have big data" problem. You know, in data science, you see people spending hours writing pandas scripts that replicate a few clicks in excel for a one of analysis. You see datasets of a few gigabytes being processed with spark when SQL would be fine. You see ML techniques being thrown at questions that could be answered simply and reliably with basi…

Talking to people is not going to help you either. You end up getting a lot of noise and making sense of what you hear is difficult. When you keep probing you will get to hear stuff thats not really critical and just often made up because you ask too many questions. Classical trap of market research.

This is true; what customers SAY they want doesn't necessarily corellate with what they will actually use or pay for.

I mean I worked on an app where in one part, the end user could upload CSV files to be used. What they SAID they wanted was basically a full data management system and RESTful API to enforce constraints, data validation, record retrieval and updating, etc. What they probably wanted was an excel sheet. I dislike how my employer was like "yeah sure if you pay for it" to them.

Re: Be good-argument-driven, not data-driven

#62

Post by [deleted] »

viridian wrote: post #46

This entire discussion makes a good case for why the general populace would benefit from being taught the basics of philosophy. In this case the topic of value is the often fraught relationship between empiricism and rationalism , and the impacts each have on the scientific process, research, education, and how we go about understanding the world. To operate with one with a complete absence of the other is to expose…

[deleted]

Re: Be good-argument-driven, not data-driven

#63

Post by safety1st »

viridian wrote: post #46

This entire discussion makes a good case for why the general populace would benefit from being taught the basics of philosophy. In this case the topic of value is the often fraught relationship between empiricism and rationalism , and the impacts each have on the scientific process, research, education, and how we go about understanding the world. To operate with one with a complete absence of the other is to expose…

I would start by simply putting everyone through a course in deductive reasoning at the earliest age possible: https://en.wikipedia.org/wiki/Deductive_reasoning

From there you can go into the whole spectrum of critical thinking approaches, and then on to what's basically the liberal arts e.g. philosophy, social sciences etc. as you desire. But the value you get from all of those things depends heavily on the framework you have for thinking about them going in.

Claiming random things are "fake news" would be a lot harder if people could work out what is and isn't fake by themselves!

Re: Be good-argument-driven, not data-driven

#64

Post by mikkergp »

crabmusket wrote: post #9

This reminds me a lot of the discussion of the scientific method by Karl Popper, and David Deutsch who was very influenced by Popper. "Being data-driven" sounds very empirical . Just look at the data, and see what you find in it. But you can't just let the data "speak for itself" without an explanation or a theory that interprets the data. Popper in Conjectures and Refutations : > Observation is always selective. It…

Doesn’t the scientific method specifically say you can’t start with the data, you have to start with a hypothesis otherwise you are subject to all sorts of selection/hindsight biases. I mean you can start with data, but then you have to develop a hypothesis and use that to create an experiment that generates new data in order to reach a conclusion. It seems like that is the compromise the author is looking for, start with a good idea, then see if you can verify it with data.

Re: Be good-argument-driven, not data-driven

#65

Post by polio »

dalbasal wrote: post #52
viridian wrote: post #46

This entire discussion makes a good case for why the general populace would benefit from being taught the basics of philosophy. In this case the topic of value is the often fraught relationship between empiricism and rationalism , and the impacts each have on the scientific process, research, education, and how we go about understanding the world. To operate with one with a complete absence of the other is to expose…

Idk how if studying philosophy helps. Most philosophers were/are themselves committed to one school or theory, with gaps galore. In any case, I think empirical science's defeat of rationalism ( eg Galileo Vs Church) has all sorry of ramifications. Social sciences like economics and psychology have a lot of trouble bridging the gaps.

Epistemology is a subfield of philosophy. Seems like a healthy understanding of that would be good for society right now.

> Most philosophers were/are themselves committed to one school or theory, with gaps galore.

Most scientists specialize one thing, but students of science don't. One can learn about many schools of philosophy, as well.

Re: Be good-argument-driven, not data-driven

#66

Post by ClumsyPilot »

verisimi wrote: post #59
viridian wrote: post #46

This entire discussion makes a good case for why the general populace would benefit from being taught the basics of philosophy. In this case the topic of value is the often fraught relationship between empiricism and rationalism , and the impacts each have on the scientific process, research, education, and how we go about understanding the world. To operate with one with a complete absence of the other is to expose…

I agree so strongly with this. The point I would add is that hardly anyone uses the empirical process directly. It is all 'this article claims this' or 'this study says that'. It's very 'meta' with little to no personal verification or testing of the claims - ie, theories based on theories or models based on models, or maps based on maps. Very few check the terrain itself to confirm that the map applies. We trust edu…

And often we have no idea who made a particular model and what are it's limitations.

Re: Be good-argument-driven, not data-driven

#67

Post by stevofolife »

shubb wrote: post #35

The related problem that I see actually more often is the "you don't have big data" problem. You know, in data science, you see people spending hours writing pandas scripts that replicate a few clicks in excel for a one of analysis. You see datasets of a few gigabytes being processed with spark when SQL would be fine. You see ML techniques being thrown at questions that could be answered simply and reliably with basi…

Why not both…

Re: Be good-argument-driven, not data-driven

#68

Post by germinalphrase »

bell-cot wrote: post #41
shubb wrote: post #35

The related problem that I see actually more often is the "you don't have big data" problem. You know, in data science, you see people spending hours writing pandas scripts that replicate a few clicks in excel for a one of analysis. You see datasets of a few gigabytes being processed with spark when SQL would be fine. You see ML techniques being thrown at questions that could be answered simply and reliably with basi…

Unfortunately, your reality-driven approach has ~zero emotional appeal for most managers, exec's, and alpha-data-scientist wanna-be's.

Why? Inadequately “technical”?

Re: Be good-argument-driven, not data-driven

#69

Post by packetlost »

shubb wrote: post #35

The related problem that I see actually more often is the "you don't have big data" problem. You know, in data science, you see people spending hours writing pandas scripts that replicate a few clicks in excel for a one of analysis. You see datasets of a few gigabytes being processed with spark when SQL would be fine. You see ML techniques being thrown at questions that could be answered simply and reliably with basi…

> You know, in data science, you see people spending hours writing pandas scripts that replicate a few clicks in excel for a one of analysis

I mean, having an Excel doc at all usually implies hour(s) of work formatting the data in structured manner. Sometimes collective decades of work depending on how much heavy lifting your 15GB .xlsx is doing.

Re: Be good-argument-driven, not data-driven

#70

Post by spaniard89277 »

ekianjo wrote: post #42
shubb wrote: post #35

The related problem that I see actually more often is the "you don't have big data" problem. You know, in data science, you see people spending hours writing pandas scripts that replicate a few clicks in excel for a one of analysis. You see datasets of a few gigabytes being processed with spark when SQL would be fine. You see ML techniques being thrown at questions that could be answered simply and reliably with basi…

Talking to people is not going to help you either. You end up getting a lot of noise and making sense of what you hear is difficult. When you keep probing you will get to hear stuff thats not really critical and just often made up because you ask too many questions. Classical trap of market research.

Go talk with your customer service. Oops, so much rotation nobody cares, everyone is cheating KPIs.
Post reply on HN