What's wrong with social science and how to fix it

What's wrong with social science and how to fix it
fantasticanachronism.com

"If the original study says an intervention raises math scores by .5 standard deviations and the replication finds that the effect is .2 standard deviations (though still significant), that is considered a success that vindicates the original study!"

Why the exclamation point here? The replication study isn't magically more accurate than the original study. If the original paper finds an 0.5 standard deviation effect and the replication study finds an 0.2 standard deviation effect, that increases our confidence that a real effect was measured, but there's no reason to believe that the replication study is more accurate than the original study. Maybe the true effect is less than measured, but maybe not. So yes, it should be considered a success.

That doesn’t mean they’re wrong, necessarily. Overcoming inertia is a huge challenge. Daunting, even.

Social science asks more of us than any other science. Physics demands that we respect electricity and don't increase the infrared opacity of the atmosphere. Chemistry requires that we not emit sulfur and nitrogen compounds into the air. But social sciences will not rarely call for the restructuring of the whole society.

This is the "problem" with social science, or more properly, with the relationship between the social sciences and the society at large. When we call for "scientific" politics, it is a relatively small ask from the natural sciences, but it is a revolution -- even the social scientists themselves use this word -- when the social sciences are included in the list (Economics is no different). Psychology, as usual, falls somewhere in between.

So the relationship between the social scientists and the politicians may never be as cordial as the relationship between the natural sciences and the politicians. The "physics envy", where social scientists lament that they do not receive the kind of deference that natural scientists do, will have to be tempered by the understanding that the cost of such deference differs widely.

(All of this is ignoring that physics had a 200-year head start)

Physics does no such thing. It tells us that increasing the heat retained in the atmosphere increases the planets surface temperature. It is a descriptive science. Not a prescriptive one. Wanting to have industrial civilization possible in the next century is why you don't increase the infrared opacity of the atmosphere. But that is a value judgment far outside the scope of physics, and one social sciences claim is theirs by right of ... something.

The metaphors people use to think about the natural world are terrible, or as Carl Sagan put it Demon-Haunted.

The reason why physics, and other hard sciences, are so useful and respected is that you can switch dependent and independent variables around with a lot of success.

If I have the ideal gas law:

PV = nRT

Then I can rearrange it and be fairly confident it still works.

P = nRT/V

If you are an engineer this is a godsend. You want to set a hard value for P but can only directly control V or T? Try the second equation! You have a chance at succeeding without having to spend decades building machines that blow up and kill everyone around them!

Politicians see that and are jealous. Surely if those lame eggheads can get things to work like that we can too. So the social sciences give you equations as well. After a bunch of statistics we see that:

time spent in school = a*wealth - c

We can't control wealth, but we can control how long people spend in school:

wealth = (time spend in school + c)/a

So if we force everyone to stay in school until they are 50 everyone will have 20 million dollars in their bank accounts.

And to anyone who asks how this works, politicians say: Why are you against science and hate poor people?