No evidence for nudging after adjusting for publication bias • Hacker News Board

Re: No evidence for nudging after adjusting for publication bias

#221

Post by runarberg » Mon, Jul 25, 2022, 5:21 PM UTC

dx034 wrote: post #219

runarberg wrote: post #134

Earlier quoted context omitted.

Can these effects be explained without inventing a new term? Because if they can then these studies didn’t really find anything did they? Whenever I see a new term being introduced as an explanation I am hesitant to accept it, as it may turn out to just be explaining the planetary motions with epicycle, when the motions can be easier explained by moving the sun to the center of the solar system instead of the earth.

Not very scientific, but isn't it just laziness? Most people (including me) are too lazy to think about all the choices they could make, so they just stay with the default choice most of the time. Not because they actually prefer it, mainly because they never even read it.

Indeed. Your alternative theory doesn’t require an extra construct, and instead uses a pretty established cognitive behavior (the tendency of inaction) to explain the same phenomenon. I would say your explanation has the advantage of Occam’s razor, whereas Nudge Theory doesn’t.

Re: No evidence for nudging after adjusting for publication bias

#222

Post by rayiner » Mon, Jul 25, 2022, 6:20 PM UTC

unityByFreedom wrote: post #193

rayiner wrote: post #188

Earlier quoted context omitted.

> You've made a lot of generalizations here that amount to, "social scientists aren't as rigorous as other areas of science, therefore we should only believe studies that disagree with their results". I think it’s more “ignore them completely.” It brings “science” into disrepute to let social science associate with the other sciences. > I don't think throwing the baby out with the bath water is helpful. There is no b…

> It brings “science” into disrepute to let social science associate with the other sciences. So you just want it renamed to "social studies" or what? What is your proposal, that nobody research this topic, or that they be separated in journals etc? I doubt that will have much impact on whether it makes the news. If you want that to change, you may need to get yourself onto the board of a journal you care about. > Th…

> So you just want it renamed to "social studies" or what? What is your proposal, that nobody research this topic, or that they be separated in journals etc

Yes. And that the rest of us stop treating it as science, citing it as science, and relying on it as science.

For example, there is a major trend in the law of treating social sciences as having truth value the way real sciences do. That’s the kind of thing we need to stop doing.

Re: No evidence for nudging after adjusting for publication bias

#223

Post by sbierwagen » Mon, Jul 25, 2022, 6:41 PM UTC

unityByFreedom wrote: post #187

sacrosancty wrote: post #168

Earlier quoted context omitted.

I think social scientists have lost the right to the benefit of the doubt. They don't preregister their trials. They don't publish their negative results. They're notoriously bad at statistics, notoriously let their political beliefs distort their conclusions, notoriously scatter their work over thousands of arbitrary, hard-to-compare small-sample-size studies instead of concentrating their resources. I don't think i…

> They don't preregister their trials. First of all, preregistration is not a requirement for the scientific method, which has functioned well for centuries. That is a recent trend in response to the overflowing amount of haphazardly published science. Second, it is up to the individual scientist to decide to preregister or not. Some social scientists may preregister. Third, small sample size may be a fair critique,…

>First of all, preregistration is not a requirement for the scientific method, which has functioned well for centuries.

As has been said before, the problem is that the scientific method is right eventually. It can and often does get stuck for decades at a time, if someone with, shall we say, durable beliefs gets tenure, amasses political power and shoves their rivals out of a field. The amyloid hypothesis is just the most recent example.

Modern metascience practices (preregistration, blinding, banning "garden of forking paths" subgroup analysis, demanding high p factors and larger n) don't replace the scientific method, they're supposed to speed it up! But, by definition, these are all political issues, so they attract political arguments.

Re: No evidence for nudging after adjusting for publication bias

#224

Post by asdff » Mon, Jul 25, 2022, 7:40 PM UTC

saalweachter wrote: post #192

asdff wrote: post #136

Earlier quoted context omitted.

I wonder how many engineers crack open a statistics book to find the correct test versus just plotting box plots and saying "see looks pretty different"

To be fair, the more profound a result the less math you need to convince anyone it is the case.

Maybe if you are parroting the result in front of investors instead of statisticians that's the case

Re: No evidence for nudging after adjusting for publication bias

#225

Post by istjohn » Mon, Jul 25, 2022, 8:06 PM UTC

unityByFreedom wrote: post #200

istjohn wrote: post #199

Earlier quoted context omitted.

It's not useful. In fact, it's actively harmful. The constant churn of findings and retractions undermines the public's belief in science. When people stop believing in science, you get things like flat earthers, global warming deniers, and vaccine skeptics.

The quibble here is whether social science is a science. Here's a good overview, including a video of Feynman which the HN crowd may appreciate: https://www.quora.com/Is-social-science-a-real-science/answe... Humanity is observable. It's just hard to collect the data. I think you can make the case that some science isn't as rigorous, or that the jury is still out, but to say that it isn't science at all is wrong in m…

I think social science is clearly possible. The question is whether what passes for social science in our scientific journals deserve the label.

Re: No evidence for nudging after adjusting for publication bias

#226

Post by IshKebab » Mon, Jul 25, 2022, 9:01 PM UTC

hackerlight wrote: post #197

IshKebab wrote: post #77

Earlier quoted context omitted.

Absence of evidence is evidence of absence. It's just not proof of absence. For example, if I search your entire house for drugs, using drug sniffing dogs and so on and I don't find any at all, that's pretty good evidence that you don't have any. It's not proof though - you might have just stashed them really well. Similarly, if people have been looking for nudge effects for ages, doing loads of studies on it for yea…

Well said. I'd just change your first sentence to "Absence of evidence can be (but isn't necessarily) evidence of absence.", which is more in line with the rest of your post.

Good point.

Re: No evidence for nudging after adjusting for publication bias

#227

Post by harry8 » Tue, Jul 26, 2022, 1:34 AM UTC

TameAntelope wrote: post #220

harry8 wrote: post #214

Earlier quoted context omitted.

So are social studies departments and funding in dire need of reform in your opinion? How does the "correctness" in your view of the original submission affect that need or non-need?

Not sure I know enough to have an opinion about that; what I've been commenting about is the "nudge is dead" attitude that I don't think is warranted.

and yet here you are responding in a thread with the conclusion clearly stated:

>means it’s past time to reform a large chunk of academia.

And you're taking exception to it, but now just claiming you're actually not, that these one sentence "you're wrong" responses are really something much more modest.

Out of interest do you work in the field? Have ties as a graduate to one of the departments? Or are you completely disinterested when assessing the research?

If I have an interest it's that quality research is performed that advances human knowledge with some kind of efficiency of the resources spend. ie fund something that is being done properly and well to some effect over something that has been shown to be run by those who seem to be utterly incompetent or fraudulent shysters or something else that engenders zero confidence.

https://www.google.com/search?q=cass+sunstein+site%3Astatmod...

Re: No evidence for nudging after adjusting for publication bias

#228

Post by unityByFreedom » Tue, Jul 26, 2022, 1:35 AM UTC

istjohn wrote: post #225

unityByFreedom wrote: post #200

Earlier quoted context omitted.

The quibble here is whether social science is a science. Here's a good overview, including a video of Feynman which the HN crowd may appreciate: https://www.quora.com/Is-social-science-a-real-science/answe... Humanity is observable. It's just hard to collect the data. I think you can make the case that some science isn't as rigorous, or that the jury is still out, but to say that it isn't science at all is wrong in m…

I think social science is clearly possible. The question is whether what passes for social science in our scientific journals deserve the label.

That is a much more reasonable question that I think would be up to the journal, and you as a reader.

Re: No evidence for nudging after adjusting for publication bias

#229

Post by unityByFreedom » Tue, Jul 26, 2022, 1:42 AM UTC

rayiner wrote: post #222

unityByFreedom wrote: post #193

Earlier quoted context omitted.

> It brings “science” into disrepute to let social science associate with the other sciences. So you just want it renamed to "social studies" or what? What is your proposal, that nobody research this topic, or that they be separated in journals etc? I doubt that will have much impact on whether it makes the news. If you want that to change, you may need to get yourself onto the board of a journal you care about. > Th…

> So you just want it renamed to "social studies" or what? What is your proposal, that nobody research this topic, or that they be separated in journals etc Yes. And that the rest of us stop treating it as science, citing it as science, and relying on it as science. For example, there is a major trend in the law of treating social sciences as having truth value the way real sciences do. That’s the kind of thing we ne…

I found the hard/soft science discussion here [1] to be informative.

It seems unlikely that all of social science will one day be declared as "not science". Aristotle's methods, for example, did not require a certain sample size.

You're applying far too strict of a definition to science. Basic forms of science can be practiced by a child at home. Journals publish more in-depth analyses, and it's up to them what to publish, at the risk or reward of gains and losses of readership.

[1] https://blogs.scientificamerican.com/cross-check/is-social-s...

Re: No evidence for nudging after adjusting for publication bias

#230

Post by unityByFreedom » Tue, Jul 26, 2022, 2:05 AM UTC

sbierwagen wrote: post #223

unityByFreedom wrote: post #187

Earlier quoted context omitted.

> They don't preregister their trials. First of all, preregistration is not a requirement for the scientific method, which has functioned well for centuries. That is a recent trend in response to the overflowing amount of haphazardly published science. Second, it is up to the individual scientist to decide to preregister or not. Some social scientists may preregister. Third, small sample size may be a fair critique,…

>First of all, preregistration is not a requirement for the scientific method, which has functioned well for centuries. As has been said before, the problem is that the scientific method is right eventually. It can and often does get stuck for decades at a time, if someone with, shall we say, durable beliefs gets tenure, amasses political power and shoves their rivals out of a field. The amyloid hypothesis is just th…

> It can and often does get stuck for decades at a time,

I think this is just more data. If we're all wrong for a longer time, then the impact will be more clear.

I agree that modern additions like preregistration are helpful. I only wanted to remark that it is not a prerequisite for science.

> by definition, these are all political issues, so they attract political arguments.

People are good at gaming systems. We are naturals at recognizing patterns and will adjust our behavior to meet our goals. In that sense, social science may be targeting a moving object, almost like the difference between observing and not observing the atoms in a double slit test.

As hard as it may be in social science, the process of hypothesizing, observing and forming conclusions is still science. For some, it appears that is not science because a definite conclusion never arrives.

Which viewpoint is correct? I think it's up to you to decide. And, when you don't grant people that choice, you get an anti-science response, because people naturally reject being told what to think. Science, for me, is about asking questions, not necessarily arriving at a definitive result.