No evidence for nudging after adjusting for publication bias

"Couldn’t this bias also happen in the other direction?" The general notion is that positive results ("we tried nudging and succeeded in getting people to behave more in X way") are more publishable than negative results ("we tried nudging, and nothing much happened"). It is a common and well-attested problem in many areas of science, but probably particularly in behavioral sciences; I have not heard of cases where publication of negative results is more likely than publication of positive results, although there are obviously heated debates over some results.

Whether publication bias is the explanation in this example, I don't pretend to know.

There's a lot of confusion here about what this article is talking about.

Nudges aren't just defaults. We've known for over a century that people are influenced by defaults. Nudges also aren't anchoring, where choices influence one another. Kahneman & Tversky won a Nobel prize for that and other behavioral economics ideas a decade before the idea of nudges.

Nudges are a bigger idea that many small changes lead to huge behavioral changes. Like providing a social reference point (see the average electricity use of your neighbors), surfacing hidden information (a red light to remind you to change your A/C filter), change the financial effort involved in something (deposit your drinking money into an account that you lose when you drink again; health plans that pay to stay healthy), change the physical effort of making bad choices (a motorcycle license for people who don't want to wear helmets that is much harder to get), change the consequences of options (pay a teenager $1/day to not get pregnant), provide reminders (check if an email is rude and have someone confirm they want to send it), public commitments (say you are doing X makes you more likely to do X), etc.

There are various examples of each of these working to some extent in specific circumstances.

But we have a lot of other tools for changing people's behavior. We have education campaigns. We have fines. We have taxes. We have tax breaks. The idea behind nudges is that they're an easy replacement for many of these other tools.

But the meta-analysis shows that nudges aren't a general-purpose tool that leads to significant changes in people's behavior. The behavioral changes are small, the same as we get from a fine, a tax, or an education campaign.

Aside from specific circumstances, nudges don't work better (and may be much worse) compared to our usual tools for getting people to behave.

"... evidence that nudging works. Perhaps not up to the standard of science..." That's pretty close to saying it doesn't work. The point of this meta-study was precisely to show that the evidence claimed to support nudging was probably attributable to random variation + unnatural selection, where the unnatural selection was publication choice: either the researchers who got negative (null) results chose not to bother writing it up and submitting it, or papers that reported negative were rejected by publishers.

There are lots of people who do X for a living, but where X doesn't work: palm readers, fortune tellers, horoscope writers, and so on. I'm not even sure that funds managers reliably obtain results much above random.

They note that there is no evidence for nudging as being generally effective. So any individual nudge could be effective (except in finance in which they found that none are effective).

We are talking about publication bias, where the decision whether to publish something is biased by the outcome of the experiment.

I think this doesn't really apply to A/B testing, because people are incentivized pay as much attention to negative results as to positive ones.