What's wrong with social science and how to fix it • Hacker News Board

Re: What's wrong with social science and how to fix it

#41

Post by closeparen » Sat, Sep 12, 2020, 5:32 AM UTC

stefan_ wrote: post #25

mhh__ wrote: post #13

I try not to look down on social science, for the most part data is data as long as you can reason about how it was collected and who by. The only thing that worries me a little (or a lot sometimes) is that there doesn't seem to be much "bone" for the meat to hang off of - that is, in physics, if your theory doesn't match experiment it's wrong whereas in social science you're never going to have a (mathematical) theo…

If only it was all data, statistics and hypotheses in social science. As it stands, that's a relatively modern idea in the field. Take a look at some of the stuff they teach in sociology 101: https://en.m.wikipedia.org/wiki/Social_actions It's all utter fiction. The long drawn out ramblings of ancient white men who formulated theories as their favorite pastime. None of it is experimentally derived because there were…

The physical sciences have no less "old white man conjecture" in their history.

Re: What's wrong with social science and how to fix it

#42

Post by stewbrew » Sat, Sep 12, 2020, 5:41 AM UTC

The "social sciences" include a lot. Wrt Sociology, I'd say one problem is the overemphasis on quantitative methods - they try to be as serious as the big boys.

The best sociological research I've read was qualitative though. Questionable replicability is of course built-in in this type of research but the research dealt with relevant questions. Most quantitative sociology seems rather irrelevant to me.

Another problem is of course that most quantitative sociologists don't have a clue what they are doing. They don't even know the basics of statistics and then use some statistical methods they don't understand. It's some kind of overcompensation, I think. Although, psychologists are even worse in this respect. It's really fun to watch an psychologists torturing SAS.

I write this as someone who was originally trained as sociologist and over the years turned into a data scientist.

Re: What's wrong with social science and how to fix it

#43

Post by dehrmann » Sat, Sep 12, 2020, 5:53 AM UTC

unishark wrote: post #40

xrisk wrote: post #38

Is there a similar study done on the physical sciences? I’m getting a bit of holier-than-thou feeling from this article. Edit: from all this talk of reproducibility, I wonder what percentage of cutting edge ML research is reproducible (either from lack of public training sets / not enough compute)

There are definitely studies criticizing ML publications similarly. As a kind of statistics (but often without the rigor), screwups make ML methods appear better than they really are. Hence the literature is packed with screwups. Other CS subfields that get a lot of criticism are "network science" and bioinformatics.

I've been playing with some ML lately. I'm sure some of this is because I only have a high level understanding of what I'm doing, but it really feels like throwing things at the problem and see what sticks and is fast. Add features, remove features, try different network architectures, different activation functions, etc. I think the best understood thing might be overfitting, which is oddly reassuring.

Re: What's wrong with social science and how to fix it

#44

Post by DrNuke » Sat, Sep 12, 2020, 6:04 AM UTC

Shameless plug with the ten relevant problems I scooped from a very recent literature review: interculturalism, introspection, truth, authenticity, human enhancement, critical thinking, technocracy, privilege, ethics, higher education. Link to free intro: https://www.tenproblems.com/2020/08/01/ten-problems-for-soci...

Re: What's wrong with social science and how to fix it

#45

Post by xrisk » Sat, Sep 12, 2020, 6:28 AM UTC

stewbrew wrote: post #42

The "social sciences" include a lot. Wrt Sociology, I'd say one problem is the overemphasis on quantitative methods - they try to be as serious as the big boys. The best sociological research I've read was qualitative though. Questionable replicability is of course built-in in this type of research but the research dealt with relevant questions. Most quantitative sociology seems rather irrelevant to me. Another probl…

I’m really interested in what you feel about the potential applications of CS/ML to sociology. Or if you might have any resources that talk about that.

I ask because I’m enrolled in a research program in “computational humanities”. My initial feeling towards the program is that it’s kind of a sham.

Computational Humanities seems to be as computational as an accountant using Excel for their work. Not that I particularly mind, I’m not very interested in the computational aspect at all.

Re: What's wrong with social science and how to fix it

#46

Post by jack_pp » Sat, Sep 12, 2020, 6:31 AM UTC

dehrmann wrote: post #43

unishark wrote: post #40

Earlier quoted context omitted.

There are definitely studies criticizing ML publications similarly. As a kind of statistics (but often without the rigor), screwups make ML methods appear better than they really are. Hence the literature is packed with screwups. Other CS subfields that get a lot of criticism are "network science" and bioinformatics.

I've been playing with some ML lately. I'm sure some of this is because I only have a high level understanding of what I'm doing, but it really feels like throwing things at the problem and see what sticks and is fast. Add features, remove features, try different network architectures, different activation functions, etc. I think the best understood thing might be overfitting, which is oddly reassuring.

Well yes. I remember hearing that there are now meta-networks, networks that optimize other networks by mechanisms you described.

Re: What's wrong with social science and how to fix it

#47

Post by SubiculumCode » Sat, Sep 12, 2020, 6:41 AM UTC

not2b wrote: post #3

This guy overstates his case somewhat. Consider: "If the original study says an intervention raises math scores by .5 standard deviations and the replication finds that the effect is .2 standard deviations (though still significant), that is considered a success that vindicates the original study!" Why the exclamation point here? The replication study isn't magically more accurate than the original study. If the orig…

Indeed, and even if the replication isn't significant, it doesn't mean that the replication and the original study significantly differ from each other.

Overall, the condescending trash talking in this article led me to flag it.

Re: What's wrong with social science and how to fix it

#48

Post by doonesbury » Sat, Sep 12, 2020, 7:04 AM UTC

mehrdadn wrote: post #36

doonesbury wrote: post #20

Earlier quoted context omitted.

Agree. In the face of institutional inertia - here crossing industry and academic fields - your starting point is when I see a fly I use a canon. It is extremely difficult to budge. As so the usual reminder: the first 51% of communication is repetition

> the first 51% of communication is repetition Is this a famous saying? It sounds nice but I hadn't heard it before. (And Google doesn't pop up anything obvious.)

That's my own turn of phrase.

Re: What's wrong with social science and how to fix it

#49

Post by rmbeard » Sat, Sep 12, 2020, 7:09 AM UTC

xrisk wrote: post #38

Is there a similar study done on the physical sciences? I’m getting a bit of holier-than-thou feeling from this article. Edit: from all this talk of reproducibility, I wonder what percentage of cutting edge ML research is reproducible (either from lack of public training sets / not enough compute)

This one comes close https://www.nature.com/news/1-500-scientists-lift-the-lid-on...

Re: What's wrong with social science and how to fix it

#50

Post by friendlybus » Sat, Sep 12, 2020, 7:14 AM UTC

Kednicma wrote: post #15

teorema wrote: post #11

Earlier quoted context omitted.

I'm not sure that psychologists really even make the distinction between "what is socially learned" and what is "inherent to humans" to be honest. I want to say no one really denies traits are influenced by social factors, but I'm sure you could find some citation to the contrary somewhere. The Big Five are pretty reproducible in part or in whole, but it's strawman to say psychologists are "never questioning whether…

You're right that I'm thinking of very basic criticisms. In particular, there's zero evidence that humans aren't p-zombies [0] and no definitive rejection of the Dodo Bird hypothesis [1]; in combination, this suggests that psychologists are both wrong to imagine that there's anything interesting going on inside of a human's mental states, and also wrong to try to classify those mental states into appropriate and inap…

There's also zero evidence that humans are p-zombies, and plenty of criticism that p-zombies don't work as a thought experiment.

It's a pretty big leap to throw away consciousness on the back of equal outcomes in psychotherapy. There are partial rejections of the Dodo bird verdict in your link.

The Cartesian theatre doesn't account for the mind's ability to imagine things that never were. As soon as you account for that via some emergent material property we have an opening to inject the properties of consciousness back into the discussion.

It's easy to say that the Big Five's cross-cultural statistical correlations are not good enough to describe people, though to dismiss it entirely off your grounding is not really going to work?

Repetition of natural structures is common. Many idealised aesthetic styles rely on that, like the Fibonacci spiral. Why would a fixed and repetitive uncertainty be any less terrifying that any other kind of uncertainty? We don't know what's before the big bang or what colour people really see in their mind when we say red.