No evidence for nudging after adjusting for publication bias • Hacker News Board

Re: No evidence for nudging after adjusting for publication bias

#101

Post by 13415 » Sun, Jul 24, 2022, 5:26 PM UTC

rjmunro wrote: post #83

Reading this, I don't get how you can take all "nudging" and declare "No evidence". Surely "nudging" encompasses a whole range of different actual actions. Some nudges work, some don't. You can't just average across all of them. I'm probably totally misunderstanding, but it sounds similar to saying "there is no evidence for medicine" because you've averaged all the papers describing medical interventions that work an…

I have the same concern. Nudging is an umbrella term for a vast number of very different activities. For example, nudging is a term used for motivating more carefully designed road markings. I find it hard to believe that some of these newer designs don't "work" better than the old ones, some of them are quite ingenious. At the same time "nudging" is also used for all sorts of public policy framing issues that are more questionable and have probably harder to measure effects. As you say, each "nudge" needs to be evaluated individually.

Re: No evidence for nudging after adjusting for publication bias

#102

Post by inglor_cz » Sun, Jul 24, 2022, 5:29 PM UTC

xyzzy123 wrote: post #60

mikkergp wrote: post #58

Earlier quoted context omitted.

Some other poster posted that they must have a pretty specific definition of nudge, because it defies credulity that defaults don’t change outcome if only because half the time I don’t read the defaults or know where to find them. I mean I just found out two weeks ago you could change the hacker news banner color. Are you telling me I’m in a statistically insignia can’t minority of hacker news users? Also how many se…

If defaults don't work then Google wasted 15 billion dollars last year paying Apple to be in the search bar... I guess there must be further detail in the paper and I will have to read it to understand the nuance.

Defaults very clearly work in matters such as consent to organ donation. In countries where you need to opt out of organ donation, few people bother to do so.

Another question is whether this increases the total amount of successful donations. I was looking around for studies and found this one [1], which basically says "in some countries, yes".

[1] https://behavioralpolicy.org/wp-content/uploads/2020/01/Does...

Re: No evidence for nudging after adjusting for publication bias

#103

Post by clairity » Sun, Jul 24, 2022, 5:32 PM UTC

JusticeJuice wrote: post #57

xyzzy123 wrote: post #46

Earlier quoted context omitted.

In the example you cited I don't think that's a nudge? Or is it? I ask because I am sure that changing defaults DEFINITELY works, especially if the user does not have a strong existing preference. You're not really changing user behaviour most of the time, you're changing the outcome of what they're trying to do, which is to reduce their cognitive load by ignoring as much as they possibly can.

Nudges are often imagined as just how choices are presented, but yes the default option is considered part of nudge theory. As also is social proof ("Your friends picked this choice"). https://en.wikipedia.org/wiki/Nudge_theory#Types_of_nudges

also, the question is how much structural elements influence outcomes (not merely decisions), not whether they do or not. that’s the extra complexity of a social system built atop a biological system built atop a chemical environment built atop a physical one. we’re complicated. physics is nigh child’s play in comparison.

Re: No evidence for nudging after adjusting for publication bias

#104

Post by mikkergp » Sun, Jul 24, 2022, 5:37 PM UTC

a_simm wrote: post #37

Wow. As a former social scientist with an axe to grind this hits hard. I like to provide that HN community with some context as to what this means. There are some 300 “research” departments in each of the major social sciences: psychology, sociology, economics and anthropology. If you believe what they say, about half of their mission is teaching and the other half is research. That’s a lot, tens of billions of dolla…

> This meta-analytic finding turns on the authors’ method for measuring publication bias. Because I accept that, I must believe that this entire body of research, probably the signal behavioral economics work, is essentially worthless!

This is a pretty short article, how are you confident of such a broad conclusion? What makes you that confident that this meta-analysis is decisive?

Re: No evidence for nudging after adjusting for publication bias

#105

Post by picardo » Sun, Jul 24, 2022, 5:38 PM UTC

13415 wrote: post #101

rjmunro wrote: post #83

Reading this, I don't get how you can take all "nudging" and declare "No evidence". Surely "nudging" encompasses a whole range of different actual actions. Some nudges work, some don't. You can't just average across all of them. I'm probably totally misunderstanding, but it sounds similar to saying "there is no evidence for medicine" because you've averaged all the papers describing medical interventions that work an…

I have the same concern. Nudging is an umbrella term for a vast number of very different activities. For example, nudging is a term used for motivating more carefully designed road markings. I find it hard to believe that some of these newer designs don't "work" better than the old ones, some of them are quite ingenious. At the same time "nudging" is also used for all sorts of public policy framing issues that are mo…

Agree with you. Nudging is a type of user experience design. UX designers nudge with every design decision they make, and the effectiveness of those decisions is quantifiable. So it's hard to argue that all nudges are ineffective, just like one can't argue that all UX design is ineffective.

Re: No evidence for nudging after adjusting for publication bias

#106

Post by adamisom » Sun, Jul 24, 2022, 5:44 PM UTC

xyzzy123 wrote: post #46

a_simm wrote: post #37

Wow. As a former social scientist with an axe to grind this hits hard. I like to provide that HN community with some context as to what this means. There are some 300 “research” departments in each of the major social sciences: psychology, sociology, economics and anthropology. If you believe what they say, about half of their mission is teaching and the other half is research. That’s a lot, tens of billions of dolla…

In the example you cited I don't think that's a nudge? Or is it? I ask because I am sure that changing defaults DEFINITELY works, especially if the user does not have a strong existing preference. You're not really changing user behaviour most of the time, you're changing the outcome of what they're trying to do, which is to reduce their cognitive load by ignoring as much as they possibly can.

Yeah, I am also confused by the statement that nudges theory doesn’t replicate and I’m afraid that statement won’t replicate haha, or rather, there are basic, indisputable findings with mindboggling effect size that countries with different defaults for organ donors have different donation rates.

Now, you can say all day long that those aren’t causal studies, but there is just no way that confounding factors like different cultures explain it, because cultures just aren’t sufficiently different, or rather cultures that are otherwise pretty similar have vastly different donation rates.

A lot of the replication crisis imo is just realizing that landmark studies were underpowered. That is, they don’t prove what they meant to prove, but that is very different from whether the effect exists i.e. an effect may exist yet be hard to prove and social scientists are rarely rigorous in study design, from training and from inherent difficulty.

Re: No evidence for nudging after adjusting for publication bias

#107

Post by oxfordmale » Sun, Jul 24, 2022, 5:45 PM UTC

lozenge wrote: post #47

oxfordmale wrote: post #9

Reading this article, it is good that the UK COVID policy wasn't based on behavioural nudging /sarcasm. The UK COVID policy heavily relied on this and one of the unwanted side effects was scaring a certain section of the population into submission. Although that may have been effective during COVID, it made it a lot harder for that segment of society to return back to normal.

Isn't locking down the obvious departure from a "nudge policy"? I've seen the claim a lot but it all goes back to documents like this one which discusses strategies for communicating to increase compliance with lockdowns. https://assets.publishing.service.gov.uk/government/uploads/... Yes, the UK government received such shocking insights as "Messaging needs to emphasise and explain the duty to protect others", and "…

There are articles like this:

https://www.theguardian.com/commentisfree/2020/mar/13/why-is...

Or from the other side of the political spectrum:

https://www.telegraph.co.uk/politics/2022/01/28/grossly-unet...

Re: No evidence for nudging after adjusting for publication bias

#108

Post by mikkergp » Sun, Jul 24, 2022, 5:46 PM UTC

it seems like what they are saying is here is some statistics that suggests that maybe there’s a bunch of missing data that would Show that nudge doesn’t have an effect. Like most science it seems like the conclusion should be, go do more research to see if we’re right, not, you should conclude that nudging doesn’t work.

Re: No evidence for nudging after adjusting for publication bias

#109

Post by light_hue_1 » Sun, Jul 24, 2022, 5:49 PM UTC

vishnugupta wrote: post #68

light_hue_1 wrote: post #14

There's a lot of confusion here about what this article is talking about. Nudges aren't just defaults. We've known for over a century that people are influenced by defaults. Nudges also aren't anchoring, where choices influence one another. Kahneman & Tversky won a Nobel prize for that and other behavioral economics ideas a decade before the idea of nudges. Nudges are a bigger idea that many small changes lead to hug…

I'm not a social scientist, so please help me understand this. The way you have defined nudge it seems like a very broad category. Some passive nudge (defaults) and some active (red lights) and a whole lot others. If a category has such a broad number of phenomenon then shouldn't we be analysing individual phenomenon instead of the category as a whole? For example; defaults may work and red-light thing may not work.…

> For example; defaults may work and red-light thing may not work. Why place them both under the same bucket at all? Why not study them in isolation?

And that's exactly how these meta analyses work! If you look in figure 1, they break down nudges both by the kind of intervention and by the domain. Maybe some types of nudges are much better than others. And maybe nudges work much better for say food vs finance.

Yes. Defaults have an effect, most other nudge types don't. But the domain doesn't matter much it seems.

Re: No evidence for nudging after adjusting for publication bias

#110

Post by _8j50 » Sun, Jul 24, 2022, 5:49 PM UTC

xyzzy123 wrote: post #46

a_simm wrote: post #37

Wow. As a former social scientist with an axe to grind this hits hard. I like to provide that HN community with some context as to what this means. There are some 300 “research” departments in each of the major social sciences: psychology, sociology, economics and anthropology. If you believe what they say, about half of their mission is teaching and the other half is research. That’s a lot, tens of billions of dolla…

In the example you cited I don't think that's a nudge? Or is it? I ask because I am sure that changing defaults DEFINITELY works, especially if the user does not have a strong existing preference. You're not really changing user behaviour most of the time, you're changing the outcome of what they're trying to do, which is to reduce their cognitive load by ignoring as much as they possibly can.

401k is a good example, I have had it for my whole career but if there was a form at any point asking me how much of my pay I want to contribute I would have said 0 because I prefer cash at hand than cash some day and all the b.s. health insurance is already taking a lot. But 401k doesn't bother me enough to change the default so I leave it be as some kind of rainy day fund. I didn't like paying the penalty to withdraw it, unless I turn 65, it will always be worth significantly less than it says on paper, I am not even convinced it is beating inflation. My point is, because people don't change the default it does not mean they have accepted it or like it, that is an incorrect conclusion.

Food is another example, I like cheese sometimes but when there is an option for it I take it out of the food most times but I won't go out of my way to ask for its removal otherwise, this has a real health impact.