Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

If you think psychological science is bad, imagine how bad it was in 1999

statmodeling.stat.columbia.edu

Post reply on HN
Print view View original

81–90 of 238 posts

Re: If you think psychological science is bad, imagine how bad it was in 1999

#81

Post by zemvpferreira »

bob_theslob646 wrote: post #65
publicola1990 wrote: post #26

Much of such dubious "science" is packaged by self help authours as miracle principles which are going to set ones life right. Popular books like "Power Of Habit" and Cal Newport's books also seem to fall into this category, and seem to enjoy huge patronage, even in HN.

May you please explain what Cal Newport has stated , that is considered "dubious science."

I'm interested as well. I agree with the overall point and do think Cal focuses too much on expert performance (ericsson et al) while completely ignoring tacit knowledge for some reason. But I've been reading his stuff for years and never thought he was one to peddle his own products beyond "this worked for me, might work for you".

In that sense I find him way better than other authors listed: he actually makes good use of the tools he recommends as a professional (as opposed to making a living spouting bullshit about other people's work).

Re: If you think psychological science is bad, imagine how bad it was in 1999

#82

Post by analog31 »

munificent wrote: post #24
m12k wrote: post #15

When you think about it, it's insane that metascience isn't studied more. We're going to question every little detail about the universe, but we're just going to take on faith that peer-review and publication in journals is an effective method for weeding out "bad" science?

> it's insane that metascience isn't studied more. It is, probably more than is apparent. See: https://en.wikipedia.org/wiki/Philosophy_of_science https://en.wikipedia.org/wiki/Sociology_of_scientific_knowle... There is an interesting recursive problem here, though: what tools do you use to scientifically analyze the scientific process? Whatever tools you use will themselves be hobbled by the same systemic flaws you…

Indeed, and if you think science is bad, imagine metascience.

Re: If you think psychological science is bad, imagine how bad it was in 1999

#83

Post by vlovich123 »

vlovich123 wrote: post #80

This is a flawed logical argument. Imagine this article from 1830: "If you think phrenology is bad, imagine how bad it was in 1810". Rinse & repeat for the appropriate time frames for alchemy, astrology, or any other field that tried to misapply science. Just because a field is studied for a long time or tries to apply the scientific method doesn't lend credence to the approach. All it means is, at best, we've manage…

To anyone downvoting me, consider this HN article from not too long ago: https://news.ycombinator.com/item?id=27489927

The general premise with these studies is that if an effect size is real, then a preliminary study would show something interesting. To my knowledge, statistically that is a nonsense argument. Small sample sizes suffer from various small sample effects to the point that you can't predict either way (otherwise there wouldn't be a point in doing a larger study). To add insult to injury, all of these kinds of studies are only on local college students, which further invalidates any potential information gleamed from a preliminary study.

TLDR: The way science is done in the social sciences is fundamentally flawed & the fact that limited funding ensures that's the case doesn't excuse that a significant enough part of the body of knowledge isn't reliable.

Re: If you think psychological science is bad, imagine how bad it was in 1999

#85

Post by Macha »

jackcosgrove wrote: post #66
dvfjsdhgfv wrote: post #49

Earlier quoted context omitted.

I don't disagree with you, but frankly would be a bit frustrating to limit one's studies of human behavior to just history without trying to understand the dynamics of current societies, trying to understand how they respond to change and so on. Both fields have a completely different set of instruments and very limited overlap.

The best social science studies involve often accidental experiments, where good experimental conditions occur not because of design but because of happenstance. The analysis of these situations could be construed as a historical case study, or it could be construed as an experiment. I agree that seeing analogues in past societies is not the best approach, but studying history can sometimes reveal experiment-like con…

Another similar issue is with data from situations that would be clearly unethical to intentionally create. Behaviour of plane crash survivors standard on mountainsides, castaways, feral children, etc

Re: If you think psychological science is bad, imagine how bad it was in 1999

#86

Post by dfxm12 »

Do we have to imagine? I'm sure many scientists in the field today were working in 1999. There's certainly enough data from the time to look at this in some level of detail, but Cherry picking an article and pointing out its flaws doesn't really prove much about the state of the field back then.

Maybe if the author mentioned that this article was highly regarded back then, there would be a point, but for all we know, the article was thought poorly of at the time and contemporary scientists just thought it slipped through the cracks.

It also doesn't talk about new controls in place today that would prevent a similarly poor article from being published, or even a system of "retracting" poor articles. I don't really trust that everything being published today is without flaw. After all, the other examples of bad science given are fairly recent.

Re: If you think psychological science is bad, imagine how bad it was in 1999

#87

Post by gregoreous »

Those dummies. They had perfectly good time machines in 1999 and they didn't even use them to visit our enlightened era. I bet they did all their calculations on some old Windows 95 system instead of investing in solid multiple core machines.

Still, they don't hold a candle to Gregor Mendel failing to incorporate DNA in his genetic work, if you can believe it.

Re: If you think psychological science is bad, imagine how bad it was in 1999

#88

Post by seph-reed »

enkid wrote: post #8
Tarsul wrote: post #2

I still don't understand why we "believed" studies that claimed knowledge about typical behavior and subconscious decisions while those studies were only based on a single experiment with limited participants, e.g. Stanford Prison Experiment. These type of studies should ALWAYS be looked at critically and not just because they fail to reproduce but because they are based on a very small sample size in a very discreet…

Tangent, but the Stanford Prison Experiment is an awful example of science. It was a researcher who wanted to prove a point and created the conditions to collect the data to prove that point. I hate that it's often the only psychological experiment many people are familiar with.

Calling it an awful example of science seems a bit extreme.

There is definitely something to learn from such an experiment, albeit not what was intended.

Re: If you think psychological science is bad, imagine how bad it was in 1999

#89

Post by dpe82 »

chromaton wrote: post #58
licebmi__at__ wrote: post #52

Earlier quoted context omitted.

That seems broad enough to describe the human history isn’t it? So, “keep doing what you do”?

Not necessarily. You could just fall back on authority, superstition, or navel-gazing.

That would probably fall under "testing [new/other] systems to see if they also result in known good values."

Re: If you think psychological science is bad, imagine how bad it was in 1999

#90

Post by Blikkentrekker »

failwhaleshark wrote: post #77
Blikkentrekker wrote: post #63

Earlier quoted context omitted.

What is a “hard science”? The only distinction I care about is exactness and non-exactness. Is the research based upon formulating a theory that is capable of forecassting not-yet observed events a nonexistent, exact margin or error, and are the conditions then re-created to see if the forecast is within the margin of error that the instruments that measure it have? Some say biology is “hard”, and some say it is “sof…

This has been answered a million times before. https://en.wikipedia.org/wiki/Hard_and_soft_science

And it starts with that the terms are colloquial and bereft o an actually hard definition.

“colloquial”, “roughly”, ”perceived”. — these are not the terms that definitions are made of.

The point is that there is no actual hard distinction between “hard science” and “soft science” but there is a hard distinction between an exact theory, and an inexact theory.

Post reply on HN