Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cargo Cult Science (1974)

calteches.library.caltech.edu

Post reply on HN
Print view View original

21–30 of 36 posts

Re: Cargo Cult Science (1974)

#21

Post by throwawayjava »

chillacy wrote: post #16
umvi wrote: post #3

Experiments that attempt to create AGI singularities by just making huge neural networks strike me as cargo cult science.

I've never seen that before but I have seen businesses: * Use hadoop/latest distributed computing with data that fits on one machine * Use way higher variance models than necessary for the problem at hand And in general just cargo cult what the big players are doing when it's unnecessarily complex for the size of the business.

s/businesses/engineers looking for resume items for FAANG applications/ and it's completely rational.

Re: Cargo Cult Science (1974)

#22

Post by throwawayjava »

seibelj wrote: post #13
aspaceman wrote: post #11

Earlier quoted context omitted.

I dislike this usage of "soft" sciences. I think it helps to be specific about the field, the journal, or the community. People use the word "soft" to describe any kind of scientific research that makes them uncomfortable, no matter the legitimacy. The social sciences are soft, psychology is soft, economics is soft, etc. It's a moving goal-post, when there are social scientists and psychologists who do very "hard" sc…

Mathematics is "hard" because there is a provably right and wrong answer. Moving away from mathematics, subjects become much more malleable and open to interpretation. For example the fuzziness of psychology allows something like the "Power Pose" to exist, while in math there would be no "Power Matrix" or some such.

A lot of applied math -- especially modeling -- is pretty freaking fuzzy.

Re: Cargo Cult Science (1974)

#23

Post by nradov »

aspaceman wrote: post #11
seibelj wrote: post #2

Out of all the experiments in the soft sciences that captured society’s imagination, one of the most egregious is “Power Posing”.[0] The idea is that you stand like a superhero in front of a mirror and all of these measurable chemical changes happen that improve your abilities. The professor that discovered this phenomena had a famous TED talk, a bestselling book, switched her faculty position to Harvard, and basical…

I dislike this usage of "soft" sciences. I think it helps to be specific about the field, the journal, or the community. People use the word "soft" to describe any kind of scientific research that makes them uncomfortable, no matter the legitimacy. The social sciences are soft, psychology is soft, economics is soft, etc. It's a moving goal-post, when there are social scientists and psychologists who do very "hard" sc…

Soft sciences are characterized by an inability to conduct controlled experiments. It's impossible to quantify all the independent variables so researchers guess or make assumptions.

Re: Cargo Cult Science (1974)

#24

Post by throwaway740077 »

aspaceman wrote: post #11
seibelj wrote: post #2

Out of all the experiments in the soft sciences that captured society’s imagination, one of the most egregious is “Power Posing”.[0] The idea is that you stand like a superhero in front of a mirror and all of these measurable chemical changes happen that improve your abilities. The professor that discovered this phenomena had a famous TED talk, a bestselling book, switched her faculty position to Harvard, and basical…

I dislike this usage of "soft" sciences. I think it helps to be specific about the field, the journal, or the community. People use the word "soft" to describe any kind of scientific research that makes them uncomfortable, no matter the legitimacy. The social sciences are soft, psychology is soft, economics is soft, etc. It's a moving goal-post, when there are social scientists and psychologists who do very "hard" sc…

They're called soft because the standards are very low when compared to the harder sciences, and it's basically impossible to do perfect controls or replications.

Re: Cargo Cult Science (1974)

#25

Post by curiousgal »

seibelj wrote: post #2

Out of all the experiments in the soft sciences that captured society’s imagination, one of the most egregious is “Power Posing”.[0] The idea is that you stand like a superhero in front of a mirror and all of these measurable chemical changes happen that improve your abilities. The professor that discovered this phenomena had a famous TED talk, a bestselling book, switched her faculty position to Harvard, and basical…

Studying Statistics has ruined every "psychology" article out there.

Re: Cargo Cult Science (1974)

#27

Post by kragen »

brandmeyer wrote: post #9
seibelj wrote: post #2

Out of all the experiments in the soft sciences that captured society’s imagination, one of the most egregious is “Power Posing”.[0] The idea is that you stand like a superhero in front of a mirror and all of these measurable chemical changes happen that improve your abilities. The professor that discovered this phenomena had a famous TED talk, a bestselling book, switched her faculty position to Harvard, and basical…

There are more subtle variations on this that still affect the harder sciences. A topic closer to home for most of us might be benchmarking. It is terribly difficult to construct an honest A/B comparison between two different algorithms, especially when the person doing the testing designed B themselves.

It's usually easy to get benchmark results that are reproducible within 10% or so, though often people don't bother. With careful work you may be able to get benchmarks reproducible to 9 significant figures. See SUPERCOP for an example.

Now, your reproducible results may or may not be fair, but that's an issue further down the line.

Re: Cargo Cult Science (1974)

#28

Post by kragen »

jeffdavis wrote: post #14

"We obviously have made no progress—lots of theory, but no progress—in decreasing the amount of crime by the method that we use to handle criminals." Not sure that's obvious. Crime has steadily fallen. Particular practices may not stand up to scrutiny, but I'm guessing a lot of it does kind of work.

In the US crime rose steadily from 1960 to 1990, and that's using the official statistics that don't even try to include things like the Iran–Contra scandal, the Gulf of Tonkin Resolution, CREEP, the Kent State murders, and the CIA protecting crack kingpins. Feynman was speaking in 1974, right in the middle of this period.

Re: Cargo Cult Science (1974)

#29

Post by kriro »

aspaceman wrote: post #11
seibelj wrote: post #2

Out of all the experiments in the soft sciences that captured society’s imagination, one of the most egregious is “Power Posing”.[0] The idea is that you stand like a superhero in front of a mirror and all of these measurable chemical changes happen that improve your abilities. The professor that discovered this phenomena had a famous TED talk, a bestselling book, switched her faculty position to Harvard, and basical…

I dislike this usage of "soft" sciences. I think it helps to be specific about the field, the journal, or the community. People use the word "soft" to describe any kind of scientific research that makes them uncomfortable, no matter the legitimacy. The social sciences are soft, psychology is soft, economics is soft, etc. It's a moving goal-post, when there are social scientists and psychologists who do very "hard" sc…

I wouldn't take this too seriously. Theoretically mathematics is also a "soft science" or no science at all depending on the science theoretical background of the person you talk to (science is only experiments with data, empirical stuff...you know, following the scientific method).

I have since developed a very soft criterion for what I call science. Roughly...stuff that advances our understanding of the world and is conducted in a rigorous way.

Re: Cargo Cult Science (1974)

#30

Post by harry8 »

Optimising software has frequently been very cargo cult-ish. No measurements at all or measurements that tell you nothing have been pretty common. It seems it's more common to take it seriously now, also tools are better and techniques get written up on blogs and shared, some of the good ones catch on, some of the bad ones eventually fall over. "If experiment doesn't match theory the theory is wrong." Feynman has been the source of a good bit of wisdom in my career and life. Re-post and re-discuss often. If you haven't, "Surely you're joking..." It's worth your time and will also entertain. I should read it again, it's been a while.
Post reply on HN