Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What's wrong with social science and how to fix it

fantasticanachronism.com

Post reply on HN
Print view View original

11–20 of 105 posts

Re: What's wrong with social science and how to fix it

#11

Post by teorema »

Kednicma wrote: post #4

> Even if all the statistical, p-hacking, publication bias, etc. issues were fixed, we'd still be left with a ton of ad-hoc hypotheses based, at best, on (WEIRD) folk intuitions. This is the quiet part which most social scientists, particularly psychologists, don't want to discuss or admit: WEIRD [0] selection bias massively distorts which effects are inherent to humans and which are socially learned. You'll hear peo…

I'm not sure that psychologists really even make the distinction between "what is socially learned" and what is "inherent to humans" to be honest. I want to say no one really denies traits are influenced by social factors, but I'm sure you could find some citation to the contrary somewhere.

The Big Five are pretty reproducible in part or in whole, but it's strawman to say psychologists are "never questioning whether personality traits are shaped by society." That's not just not true, nor is it even clear what that question means. Go to Google Scholar and search for "Big Five" and terms like "measurement invariance" or "cultural" or "social" or "societies" and take a look.

The Big Five are meant to be descriptive, the "why" is a different issue. (Just to explain it a different way, let's say you do unsupervised learning of cat images, and find over and over and over and over and over again over decades and different databases that the algorithms always return the same 5 types of cats, plus or minus a little. Wouldn't you make a note of it if you were interested in visual types of cats?) And it's important to remember that some consensus around the Big Five wasn't really until the 90s (even today I'm not even sure there's "consensus" around the Big Five).

I agree that there's a problem with selection of participants, but the only way to do that is to increase participation of the scientific community worldwide. And there are whole fields (cultural psychology) dedicated to the problems surrounding this issue.

The Freudian comparison is also worth commenting on in two respects: first, Freudians got in trouble for not pursuing falsifiable empirical research, which is simply not the case for the things you're talking about. Second, everyone loves to hate on Freud, but the basic tenets of unconscious versus conscious processes that sometimes conflict are still a bedrock of neurobehavioral research, including two-system theories ("fast and slow"), which won someone a Nobel prize and is a darling of cognitive researchers. There are legitimate discussions to be have about the utility of two-system theories but those discussions are far more sophisticated than the criticisms I think you're referring to.

Re: What's wrong with social science and how to fix it

#12

Post by teorema »

scythe wrote: post #7

It's common to see this topic: what's "wrong" with social science. But there are always some things wrong with every science. If nothing was wrong, there wouldn't be any science left to do. Social science asks more of us than any other science. Physics demands that we respect electricity and don't increase the infrared opacity of the atmosphere. Chemistry requires that we not emit sulfur and nitrogen compounds into t…

Social scientists turn the microscope on themselves also. When the microscope turns elsewhere you see similar patterns to differing extents (cf. recent articles on reanalysis of fMRI data, pharmacology replication rates, Theranos or hydroxychloroquine).

Meta-science has always been the gift of social science. This will all eventually funnel down elsewhere, just like meta-analysis.

But you're right, in that social science hits very close to home, more so than other sciences. Imagine that it suddenly worked very very well, and someone in the field of neuropsychology could manipulate behavior just like you might a lightbulb. Isn't that what critics are really asking for?

Re: What's wrong with social science and how to fix it

#13

Post by mhh__ »

I try not to look down on social science, for the most part data is data as long as you can reason about how it was collected and who by.

The only thing that worries me a little (or a lot sometimes) is that there doesn't seem to be much "bone" for the meat to hang off of - that is, in physics, if your theory doesn't match experiment it's wrong whereas in social science you're never going to have a (mathematical) theory like that so you have to start (in effect) guessing. The data is really muddy, but thanks to recent (good) political developments whatever conclusions can be drawn from it may not be right in their eyes. For example, (apparently) merely commenting on the variation hypothesis can get you fired [https://en.wikipedia.org/wiki/Variability_hypothesis#Contemp...].

Re: What's wrong with social science and how to fix it

#15

Post by Kednicma »

teorema wrote: post #11
Kednicma wrote: post #4

> Even if all the statistical, p-hacking, publication bias, etc. issues were fixed, we'd still be left with a ton of ad-hoc hypotheses based, at best, on (WEIRD) folk intuitions. This is the quiet part which most social scientists, particularly psychologists, don't want to discuss or admit: WEIRD [0] selection bias massively distorts which effects are inherent to humans and which are socially learned. You'll hear peo…

I'm not sure that psychologists really even make the distinction between "what is socially learned" and what is "inherent to humans" to be honest. I want to say no one really denies traits are influenced by social factors, but I'm sure you could find some citation to the contrary somewhere. The Big Five are pretty reproducible in part or in whole, but it's strawman to say psychologists are "never questioning whether…

You're right that I'm thinking of very basic criticisms. In particular, there's zero evidence that humans aren't p-zombies [0] and no definitive rejection of the Dodo Bird hypothesis [1]; in combination, this suggests that psychologists are both wrong to imagine that there's anything interesting going on inside of a human's mental states, and also wrong to try to classify those mental states into appropriate and inappropriate states. Instead, what ends up getting studied is society's own idea of what ought to be happening inside our homoncular Cartesian theater [2].

Given these foundational issues, it's folly to try to support Big Five or any other descriptive model just by saying that it's a good fit for the numbers. Any principal component analysis will find something which factors out as if it were a correlative component. This dooms Big Five just as reliably as it dooms g-factors or Myers-Briggs or any other astrology-like navel-gazing.

(If you want an example of actual five things showing up again and again and again, mathematics has examples [3][4][5], but it turns out that when actual five things show up, then the reaction is not to serenely admire the correlation, but to admit terror before cosmic uncertainty. Psychologists do not seem to go insane and kill themselves like statistical mechanics or set theorists; have they really seen the face of god?)

[0] https://en.wikipedia.org/wiki/Philosophical_zombie

[1] https://en.wikipedia.org/wiki/Dodo_bird_verdict

[2] https://en.wikipedia.org/wiki/Cartesian_theater

[3] https://en.wikipedia.org/wiki/ADE_classification

[4] https://en.wikipedia.org/wiki/Monstrous_moonshine

[5] https://en.wikipedia.org/wiki/Classification_of_finite_simpl...

[6] https://en.wikipedia.org/wiki/Hard_problem_of_consciousness

Re: What's wrong with social science and how to fix it

#16

Post by harmful_stereo »

konjin wrote: post #10
Kednicma wrote: post #4

> Even if all the statistical, p-hacking, publication bias, etc. issues were fixed, we'd still be left with a ton of ad-hoc hypotheses based, at best, on (WEIRD) folk intuitions. This is the quiet part which most social scientists, particularly psychologists, don't want to discuss or admit: WEIRD [0] selection bias massively distorts which effects are inherent to humans and which are socially learned. You'll hear peo…

The people who use Hanlon's razor to explain away malice are both incompetent and malicious. Only someone who is an idiot would ever think to use 'I'm very stupid' as an excuse or explanation why they did something very damaging. If you are smart enough to realize you are incompetent after the fact you were smart enough to realize it before the fact, and that means you were malicious in not recusing yourself.

The way this was stated in one discrete thought leads me to a problem with human nature i dwell on: how much of what our society and culture is, and what authority is, is just a effort to disguise our frailty and fallibility? It is tremendously hard to be reliable and competitive across multiple disciplines and for the majority of tasks involved in basic human life. There is too big a trade off between available time and location and doing any task well. We are primarily hunting-gathering the easiest ways to meet our needs. How can you blame people for not recusing themselves from participating or misrepresenting themselves as competent when our culture values that and expresses it so dramatically at the highest levels of public performance,from IPOs to high office and everywhere else. Storytelling in the tradition of the heroic myth is mostly about becoming qualified to assume a social role, as an upward stuggle.

It seems built into human character to bite off far more than we can chew, as in free real estate, and then leverage the social value of holding something others are willing to compete for. I think it amounts to a social survival instinct, and i lament how there's very little chance of discouraging people from doing it because of the potential payoff. If anything i think it's a failure of institutions for being built to exploit that competition rather than guard against its excesses.

Re: What's wrong with social science and how to fix it

#19

Post by tomrod »

konjin wrote: post #9
scythe wrote: post #7

It's common to see this topic: what's "wrong" with social science. But there are always some things wrong with every science. If nothing was wrong, there wouldn't be any science left to do. Social science asks more of us than any other science. Physics demands that we respect electricity and don't increase the infrared opacity of the atmosphere. Chemistry requires that we not emit sulfur and nitrogen compounds into t…

>Physics demands that we respect electricity and don't increase the infrared opacity of the atmosphere. Physics does no such thing. It tells us that increasing the heat retained in the atmosphere increases the planets surface temperature. It is a descriptive science. Not a prescriptive one. Wanting to have industrial civilization possible in the next century is why you don't increase the infrared opacity of the atmos…

This is why knowledge of causal inference is essential.

Causality is not established via tweaking a correlation or regression analysis, and we social scientists should know that.

Re: What's wrong with social science and how to fix it

#20

Post by doonesbury »

hinkley wrote: post #5
not2b wrote: post #3

This guy overstates his case somewhat. Consider: "If the original study says an intervention raises math scores by .5 standard deviations and the replication finds that the effect is .2 standard deviations (though still significant), that is considered a success that vindicates the original study!" Why the exclamation point here? The replication study isn't magically more accurate than the original study. If the orig…

If someone is selling you on rethinking an entire discipline they are probably overstating their case a bit. Or you’re ignoring them outright. That doesn’t mean they’re wrong, necessarily. Overcoming inertia is a huge challenge. Daunting, even.

Agree. In the face of institutional inertia - here crossing industry and academic fields - your starting point is when I see a fly I use a canon. It is extremely difficult to budge. As so the usual reminder: the first 51% of communication is repetition
Post reply on HN