Live data from Hacker News

No evidence for nudging after adjusting for publication bias

pnas.org

161–170 of 246 posts

Re: No evidence for nudging after adjusting for publication bias

#161
Anyone who has ever run an installer, that has an annoying setting hidden in the default-toggled-off-advanced section, knows that nudges do work. Perhaps nudges do not work on you, but when you see your parent's computer with all the desktop icons and browser toolbars in days gone bye, you will see the evidence of them working.

Re: No evidence for nudging after adjusting for publication bias

#162
post #110

Earlier quoted context omitted.

401k is a good example, I have had it for my whole career but if there was a form at any point asking me how much of my pay I want to contribute I would have said 0 because I prefer cash at hand than cash some day and all the b.s. health insurance is already taking a lot. But 401k doesn't bother me enough to change the default so I leave it be as some kind of rainy day fund. I didn't like paying the penalty to withdr…

If you’re worried about beating inflation, have a long time horizon, and don’t mind some risk you might looking into investing in total market index funds. An index fund for the S&P 500 has averaged ~10% returns when looking back 30 years [1]. If you’re just concerned about inflation, don’t like risk, and don’t mind locking you money up for a little bit Treasury Inflation Protected Securities [2] are also a thing. Th…

To add to the above, here's a graph of inflation adjusted S&P 500 [1].

[1] https://www.multpl.com/inflation-adjusted-s-p-500

Re: No evidence for nudging after adjusting for publication bias

#163

Reading through the article, it seems the actual claim is much, much weaker than the title: > However, all intervention categories and domains apart from “finance” show evidence for heterogeneity, which implies that some nudges might be effective, even when there is evidence against the mean effect So the article is saying when you look at studies of all "nudges" as whole, adjust for publication bias[0], there isn't…

> There's no way you could reasonably make the claim that no diets work.

Even if some studies results suggest that diets work, that does not, by itself, mean we should reject the null hypothesis (that diets don't work).

Coincidences exist: https://xkcd.com/882/

Re: No evidence for nudging after adjusting for publication bias

#164
post #8

And a million advertisers and designers cry out. We should spread this widely in the hope that the pop ups and banners die off.

This is why I think nudging must work on some level. What are trends, advertising, fads, etc. if not nudges? Is the existence of an intelligent nudger required or can the hive nudge itself?

> This is why I think nudging must work on some level.

Or just that people want to believe it does?

Re: No evidence for nudging after adjusting for publication bias

#165

Earlier quoted context omitted.

Some other poster posted that they must have a pretty specific definition of nudge, because it defies credulity that defaults don’t change outcome if only because half the time I don’t read the defaults or know where to find them. I mean I just found out two weeks ago you could change the hacker news banner color. Are you telling me I’m in a statistically insignia can’t minority of hacker news users? Also how many se…

I'm guilty of not reading this paper in any detail but it feels that the default setting "nudge" idea should work as described. So if you e.g. nudge people by setting up a pension plan by default (that they can opt out of) does that seriously fail to cause more people to have a pension? Or is this claiming something else?

Another similar example is jurisdictions that switched to assuming an individual consents to organs donation when they die, rather than having an opt-in system, see much higher rates of organ donation.

https://sparq.stanford.edu/solutions/opt-out-policies-increa...

Re: No evidence for nudging after adjusting for publication bias

#166
post #7

Remember, Obama appointed a regulatory czar based on this science: https://www.nytimes.com/2010/05/16/magazine/16Sunstein-t.htm...

Obama would have appointed Sunstein no matter what book he wrote, because he was one of UChicago's superstar professors (and was at the same time being aggressively courted by schools around the country; he ended up, of course, at Harvard). His appointment was notable not so much for "nudges" as for the fact that Sunstein was probably one of the most conservative appointments Obama made.

Re: No evidence for nudging after adjusting for publication bias

#167
post #37

Wow. As a former social scientist with an axe to grind this hits hard. I like to provide that HN community with some context as to what this means. There are some 300 “research” departments in each of the major social sciences: psychology, sociology, economics and anthropology. If you believe what they say, about half of their mission is teaching and the other half is research. That’s a lot, tens of billions of dolla…

Isn't this just one specific analysis using a very narrow definition of "nudge", one that doesn't even begin to encompass the work being done at those "300 "research" departments"? Further, isn't this using the same data from the original meta analysis that did find "of small to medium size" effect[0]? Why would this, alone, undo decades of research and clear, bright-line conclusions such as the ones cited in my sibl…

I agree, I have worked at 5 separate tech companies and have conducted hundreds of statistically significant experiments, changing what people select by default. These methods have been effective at helping hundreds of millions of people improve choices. It’s common practice, think: pricing on Amazon, default tips on Uber, default purchase price in a video game, etc.

I don’t see how this would make me reinterpret all those successful results. Maybe I don’t understand what this is saying.

Re: No evidence for nudging after adjusting for publication bias

#168
post #37

Wow. As a former social scientist with an axe to grind this hits hard. I like to provide that HN community with some context as to what this means. There are some 300 “research” departments in each of the major social sciences: psychology, sociology, economics and anthropology. If you believe what they say, about half of their mission is teaching and the other half is research. That’s a lot, tens of billions of dolla…

Isn't this just one specific analysis using a very narrow definition of "nudge", one that doesn't even begin to encompass the work being done at those "300 "research" departments"? Further, isn't this using the same data from the original meta analysis that did find "of small to medium size" effect[0]? Why would this, alone, undo decades of research and clear, bright-line conclusions such as the ones cited in my sibl…

I think social scientists have lost the right to the benefit of the doubt. They don't preregister their trials. They don't publish their negative results. They're notoriously bad at statistics, notoriously let their political beliefs distort their conclusions, notoriously scatter their work over thousands of arbitrary, hard-to-compare small-sample-size studies instead of concentrating their resources. I don't think it matters if this criticism is right or not. The fact that it's even possible to make such a criticism is already a condemnation of the scientific incompetence of social scientists.

Re: No evidence for nudging after adjusting for publication bias

#169

Earlier quoted context omitted.

If defaults don't work then Google wasted 15 billion dollars last year paying Apple to be in the search bar... I guess there must be further detail in the paper and I will have to read it to understand the nuance.

Defaults very clearly work in matters such as consent to organ donation. In countries where you need to opt out of organ donation, few people bother to do so. Another question is whether this increases the total amount of successful donations. I was looking around for studies and found this one [1], which basically says "in some countries, yes". [1] https://behavioralpolicy.org/wp-content/uploads/2020/01/Does...

I've heard people argue that that effect isn't a nudge, it's deceit.

That is, all you're doing is tricking people who didn't read carefully. People don't know they've opted in and would opt out if you called and told them that they checked the box.

I find it generally plausible that defaults don't matter much for what people consider very important decisions. I have minimal experience in this area, though.

Re: No evidence for nudging after adjusting for publication bias

#170
post #37

Wow. As a former social scientist with an axe to grind this hits hard. I like to provide that HN community with some context as to what this means. There are some 300 “research” departments in each of the major social sciences: psychology, sociology, economics and anthropology. If you believe what they say, about half of their mission is teaching and the other half is research. That’s a lot, tens of billions of dolla…

> Based on the nudge findings, after much debate and effort, they were updated to dictate that the max options forms was pre selected in the brief it would cause more individuals would opt for that as opposed to contributing zero.

I would be shocked if that wasn't true though. Is there any evidence it's not true in that specific case, that pre-selecting the max options causes more individuals to opt for that? Have individuals opting for that indeed gone up since this was done?

Post reply on HN