Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Be good-argument-driven, not data-driven

twitchard.github.io

Post reply on HN
Print view View original

121–130 of 168 posts

Re: Be good-argument-driven, not data-driven

#121

Post by [deleted] »

crabmusket wrote: post #20
guerrilla wrote: post #11

Earlier quoted context omitted.

> Observation is always selective. It needs a chosen object, a definite task, an interest, a point of view, a problem. And its description presupposes a descriptive language ... which in its turn presupposes interests, points of view, and problems. Thanks, I'd never heard this quote before. He's pretty much describing pragmatism à la William James. I had no idea.

The pragmatists went a little bit too far in my opinion, though it has been a long time since I read any of them. Popper is describing observations, not reality. I highly recommend Conjectures if you can find a copy. It's a short read and interesting.

[deleted]

Re: Be good-argument-driven, not data-driven

#122

Post by throwaway0asd »

NateEag wrote: post #45
throwaway0asd wrote: post #31

A major exception to this reasoning is performance. Argument driven performance suggestions are wrong more than 80% of the time and likely wrong by several orders of magnitude. You can’t know just how wrong you are without appropriate data. This makes for a good litmus test of whether people are lying to you about software or, more likely, have absolutely no idea what they are doing.

Performance falls into the article's category of "things you can reliably measure." Thus, the author would agree that in performance optimization, you should collect and analyze data.

The problem isn’t what the article author believes, but rather what developers commonly (perhaps almost universally) believe.

Most developers will fall back to intuition for any performance oriented decision even when they otherwise prefer data oriented decisions and even when the task at hand is critical to the health of their product/business. This is because performance measures require:

1. Additional effort

2. (most importantly) A willingness to abandon familiar concepts of approach

Sometimes such decisions vested in intuition are truth by omission, a form of lying, because the resulting self-comfort is worth more than the numeric benefits.

Re: Be good-argument-driven, not data-driven

#123

Post by musingsole »

BeFlatXIII wrote: post #88
mikkergp wrote: post #64

Earlier quoted context omitted.

Doesn’t the scientific method specifically say you can’t start with the data, you have to start with a hypothesis otherwise you are subject to all sorts of selection/hindsight biases. I mean you can start with data, but then you have to develop a hypothesis and use that to create an experiment that generates new data in order to reach a conclusion. It seems like that is the compromise the author is looking for, start…

The scientific method as taught in K–12 schools is largely pablum. Often, the real process (beyond iterating off prior research) begins with collecting data, then noticing patterns to make a hypothesis to be tested with targeted data collection.

A lot of this perspective depends on what point in time you choose as the start of the process. You can start with the hypothesis, or you can start with what gave rise to the hypothesis: exploration.

But, it's a layman's mistake to confuse the two and use it as a critique of the formalized scientific method.

Science bodies (like the NIH) explicitly forbid reuse or reinterpretation of data. An individual may use exploration as inspiration for a hypothesis...but for it grow into science out of curiosity requires new data generation from a carefully considered framework for the hypothesis.

Re: Be good-argument-driven, not data-driven

#124

Post by platz »

viridian wrote: post #46

This entire discussion makes a good case for why the general populace would benefit from being taught the basics of philosophy. In this case the topic of value is the often fraught relationship between empiricism and rationalism , and the impacts each have on the scientific process, research, education, and how we go about understanding the world. To operate with one with a complete absence of the other is to expose…

Most argumentation we do as on questions that are worth debating aren't based purely on deductive reasoning, but more on informal reasoning and heuristics with limited evidence.

Toulmin identifies the three essential parts of any argument as

    - the claim
    - the data (also called grounds or evidence), which support the claim
    - the warrant.
The warrant is the assumption on which the claim and the evidence depend. Another way of saying this would be that the warrant explains why the data support the claim.

Toulmin says that the weakest part of any argument is its weakest warrant. Remember that the warrant is the link between the data and the claim. If the warrant isn’t valid, the argument collapses.

Example:

    Claim: You should buy our toothwhitening product.
    Data or Grounds: Studies show that teeth are 50% whiter after using the product for a specified time.
    Warrant: People want whiter teeth.
Notice that those commercials don’t usually bother trying to convince you that you want whiter teeth; instead, they assume that you have accepted the value our culture places on whiter teeth.

https://www.blinn.edu/writing-centers/pdfs/Toulmin-Argument....

Re: Be good-argument-driven, not data-driven

#125

Post by Lwepz »

safety1st wrote: post #63
viridian wrote: post #46

This entire discussion makes a good case for why the general populace would benefit from being taught the basics of philosophy. In this case the topic of value is the often fraught relationship between empiricism and rationalism , and the impacts each have on the scientific process, research, education, and how we go about understanding the world. To operate with one with a complete absence of the other is to expose…

I would start by simply putting everyone through a course in deductive reasoning at the earliest age possible: https://en.wikipedia.org/wiki/Deductive_reasoning From there you can go into the whole spectrum of critical thinking approaches, and then on to what's basically the liberal arts e.g. philosophy, social sciences etc. as you desire. But the value you get from all of those things depends heavily on the framewor…

>I would start by simply putting everyone through a course in deductive reasoning at the earliest age possible

Indeed. This would help ensure people's brains' transition function is stable enough to perform faultless computation. We forget that our brains aren't wired for exact computation. They're wired to perform approximations of computation that are good enough for survival.

As a result, you end up with myriads students who go through the school system via memorization and emergent fuzzy computation.

They reach an adult age without possessing the cognitive tool-set to grasp the subtleties and nuances of the world they live in. The fact that such people are also preyed on by charlatans, ad companies and politicians(intersection of charlatans and ad companies) obviously doesn't help.

Re: Be good-argument-driven, not data-driven

#126

Post by majormajor »

packetlost wrote: post #69
shubb wrote: post #35

The related problem that I see actually more often is the "you don't have big data" problem. You know, in data science, you see people spending hours writing pandas scripts that replicate a few clicks in excel for a one of analysis. You see datasets of a few gigabytes being processed with spark when SQL would be fine. You see ML techniques being thrown at questions that could be answered simply and reliably with basi…

> You know, in data science, you see people spending hours writing pandas scripts that replicate a few clicks in excel for a one of analysis I mean, having an Excel doc at all usually implies hour(s) of work formatting the data in structured manner. Sometimes collective decades of work depending on how much heavy lifting your 15GB .xlsx is doing.

Often that work exists regardless of if a table of processed data that engineering formatted and schema-fied is dumped out to Excel or queried over SQL into Pandas...

I've seen this myself: the person who "naively" downloads that table and plays around in excel finds interesting things that the person who was using Pandas hadn't, because the code to manipulate columns and do certain types of calcs is actually more time consuming to write and modify than making a bunch of new columns in Excel with a bunch of formulas!

A good data scientist will have a more rigorous approach to their notebooks and practice reuse and so on... but that's not necesssarily easy.

Re: Be good-argument-driven, not data-driven

#127

Post by kylereeve »

bell-cot wrote: post #101
robertlagrant wrote: post #90

Earlier quoted context omitted.

Data has CYA appeal.

Needing to CYA also has pretty low emotional appeal for managers, exec's, and alpha-data-scientist wanna-be's. (Until it's just about too late, obviously.) And recall Mark Twain's old quip about lies & statistics. The more & bigger data that the folks who control the data & analysis have, the easier it is to make sure that those meet their own emotional & political needs.

Wasn't that Will Rogers?

Re: Be good-argument-driven, not data-driven

#128

Post by jpeloquin »

musingsole wrote: post #123
BeFlatXIII wrote: post #88

Earlier quoted context omitted.

The scientific method as taught in K–12 schools is largely pablum. Often, the real process (beyond iterating off prior research) begins with collecting data, then noticing patterns to make a hypothesis to be tested with targeted data collection.

A lot of this perspective depends on what point in time you choose as the start of the process. You can start with the hypothesis, or you can start with what gave rise to the hypothesis: exploration. But, it's a layman's mistake to confuse the two and use it as a critique of the formalized scientific method. Science bodies (like the NIH) explicitly forbid reuse or reinterpretation of data. An individual may use explo…

> Science bodies (like the NIH) explicitly forbid reuse or reinterpretation of data.

I think your main point is that collecting new data is necessary to test existing ideas. But reuse and reinterpretation of data is routine, e.g., in meta-analyses. It's not forbidden. You do have to disclose where the data came from.

Re: Be good-argument-driven, not data-driven

#129

Post by wtetzner »

TimTheTinker wrote: post #120
jkingsbery wrote: post #74

Earlier quoted context omitted.

I think there are lots of reasons why. One possible reason: no one whose job it is to write Python scripts was ever promoted for making an Excel spreadsheet when that is the simpler and more practical approach. And no manager of people who write Python scripts is going to be able to use that Excel spreadsheet to sell "I need more responsibility and head count." People tend to follow incentives, rather than focusing o…

> People tend to follow incentives, rather than focusing on making wise decisions. This is the key issue. Solving it isn't easy -- it requires people who are wise, and wisdom is a scarce commodity.

Even wise people likely follow the incentives. What is wise about doing something that your employer doesn’t reward in exchange for doing something that they will reward?

Re: Be good-argument-driven, not data-driven

#130

Post by coldtea »

dxbydt wrote: post #84
shubb wrote: post #35

The related problem that I see actually more often is the "you don't have big data" problem. You know, in data science, you see people spending hours writing pandas scripts that replicate a few clicks in excel for a one of analysis. You see datasets of a few gigabytes being processed with spark when SQL would be fine. You see ML techniques being thrown at questions that could be answered simply and reliably with basi…

frankly, there’s only a tiny handful of these mythical saas “1000 users each paying 1 mullion dollars” companies. the vast, vast majority of saas startups are serving millions of “users” - i put that in quotes because these aren’t real users or customers. they are real people checking out your product - but they aren’t users or customers. if you set up a gas station near the off ramp of some major interstate, say I-6…

>the vast, vast majority of saas startups are serving millions of “users”

There are tons of B2B saas, including regional ones, that only serve a small number of customers way under millions.

Post reply on HN