Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

No evidence for nudging after adjusting for publication bias

pnas.org

Post reply on HN
Print view View original

71–80 of 246 posts

Re: No evidence for nudging after adjusting for publication bias

#71

Post by pessimizer »

xyzzy123 wrote: post #60
mikkergp wrote: post #58

Earlier quoted context omitted.

Some other poster posted that they must have a pretty specific definition of nudge, because it defies credulity that defaults don’t change outcome if only because half the time I don’t read the defaults or know where to find them. I mean I just found out two weeks ago you could change the hacker news banner color. Are you telling me I’m in a statistically insignia can’t minority of hacker news users? Also how many se…

If defaults don't work then Google wasted 15 billion dollars last year paying Apple to be in the search bar... I guess there must be further detail in the paper and I will have to read it to understand the nuance.

Or Google is using the search bar as a pretense for paying Apple for something else.

Re: No evidence for nudging after adjusting for publication bias

#72

Post by pessimizer »

civilized wrote: post #51
pessimizer wrote: post #42

Earlier quoted context omitted.

I think some people careen from fully trusting one thing to fully trusting the opposite thing. If you're not one of those people, you'll never understand dismissing things on the basis of "you should be critical of this, because not everything that people say is true." I do feel like that, even though being critical is something we should always do, that in cases where 1) the only reason you started paying attention…

There's a lot of junk behavioral science out there, but things like "people often go with a default or recommended option so they can move on with their day" seem so obvious to me that I become suspicious of this debunking for debunking too much.

That's just refusing to be convinced by evidence, though. It's good to have hunches, but it's good to let them go after you've done the experiments. Come up with a new hunch and a new experiment that shows why the expected effects weren't seen, and you're right back in there.

Re: No evidence for nudging after adjusting for publication bias

#73

Post by solarkraft »

JusticeJuice wrote: post #66
civilized wrote: post #17

I agree with other commenters that it's unlikely nudges never have an impact. We should also be wary of high-profile debunkings, now that they're increasingly in fashion due to the replication crisis and the general dour mood. It's easy to p-hack a result into significance, but you can just as easily hack results into insignificance. These days, both findings and debunkings need a skeptical eye.

I'm a UI designer, and my experience of implementing 'nudges' is that sometimes they work, and sometimes they don't. The reality is that the way people make decisions is stupidly complex, because people have stupidly complex lives. Some tweak will work great for one project, and do nothing on the next one. It's hard to even say if it was the nudge that worked the first time. I really view nudge theory as one of many…

I'd love to read more about this. Do you have examples?

Re: No evidence for nudging after adjusting for publication bias

#74

Post by pessimizer »

akyu wrote: post #30
mcswell wrote: post #16

Earlier quoted context omitted.

"... evidence that nudging works. Perhaps not up to the standard of science..." That's pretty close to saying it doesn't work. The point of this meta-study was precisely to show that the evidence claimed to support nudging was probably attributable to random variation + unnatural selection, where the unnatural selection was publication choice: either the researchers who got negative (null) results chose not to bother…

>That's pretty close to saying it doesn't work. No it's really not. To say things a different way, I don't think this study will change anything for people actually doing choice architecture in applied settings. They have results that speak for themselves.

> results that speak for themselves.

This is exactly how a midwife explained to me why she uses magic crystals. She told me that there's science, and there's results, and that she's seen the crystals work.

Re: No evidence for nudging after adjusting for publication bias

#75

Post by mikkergp »

mcswell wrote: post #12
mikkergp wrote: post #2

If publication bias is the exclusion of publishing results that doesn’t support your hypothesis, how are they taking that into account? If I’m interpreting this correctly(and I by no means am sure that I am), I infer that they are saying in a fair publishing environment you’d expect to see more results that show less decisive results, therefor the current set of results is likely biased. Couldn’t this bias also happe…

"Couldn’t this bias also happen in the other direction?" The general notion is that positive results ("we tried nudging and succeeded in getting people to behave more in X way") are more publishable than negative results ("we tried nudging, and nothing much happened"). It is a common and well-attested problem in many areas of science, but probably particularly in behavioral sciences; I have not heard of cases where p…

I guess the question would be are positive results missing for another reason, like they are harder to test for, therefor the data looks better than it should because in the aggregate they are better, but yeah this is probably unlikely.

Re: No evidence for nudging after adjusting for publication bias

#76

Post by wawjgreen »

There's a little-known theory (whose name eludes me) that states: any outcome in behaviour is highly dependent on the immediate and unpredictable interplay of various environment variables and their real and perceived effect on the person. this alone cancels the efficacy of any nudges (but some variables may be in the same direction as the nudge--hence the original but misguided nudge research: they were fooled by randomness). I have seen married women who were loyal to their husbands (and who had no idea they would fall for a guy who practiced "seduction" on them) become bewildered and surprised by their own behaviour even though the behavior went against their firmly-held opinions about themselves (that is: I love my husband and I am loyal to him.). The environmental variables used by the person (who is a marketer) were too strong for their opinions to hold out against. As an example, he would invite them to his studio, which he had decorated (and clean) and made it so homely and snug and comfy that the first lines of defense were broken before they had a chance to realize what trap they were in. The marketer also tried to brainwash me (but failed) because I knew the power of variables and this knowledge alone saved me--even though he seemed irresistibly charming in the capacity of a father figure I never had.

Re: No evidence for nudging after adjusting for publication bias

#77

Post by IshKebab »

hackerlight wrote: post #5
picardo wrote: post #3

I'm not very familiar with how "nudging" is defined in the behavioral economics, and perhaps someone can enlighten me, but personally I find it hard to believe that it can be disproven that the way a choice is presented doesn't play any role in one's decision. The Goldilock principle is well-known. Most people instinctively choose the middle option, when given a three things to choose from. Does this study imply that…

You are misunderstanding. They're saying that there's an absence of evidence, not evidence of absence.

Absence of evidence is evidence of absence. It's just not proof of absence.

For example, if I search your entire house for drugs, using drug sniffing dogs and so on and I don't find any at all, that's pretty good evidence that you don't have any. It's not proof though - you might have just stashed them really well.

Similarly, if people have been looking for nudge effects for ages, doing loads of studies on it for years, and none of them have found any effect, then that's pretty good evidence that the effects don't exist. It's not proof though; they might just have not been very good experiments.

Re: No evidence for nudging after adjusting for publication bias

#78

Post by kingkawn »

a_simm wrote: post #37

Wow. As a former social scientist with an axe to grind this hits hard. I like to provide that HN community with some context as to what this means. There are some 300 “research” departments in each of the major social sciences: psychology, sociology, economics and anthropology. If you believe what they say, about half of their mission is teaching and the other half is research. That’s a lot, tens of billions of dolla…

Does academic social science research have credibility to damage?

Re: No evidence for nudging after adjusting for publication bias

#79

Post by [deleted] »

ghaff wrote: post #45
tgv wrote: post #28

Earlier quoted context omitted.

> A nudge, according to Thaler and Sunstein is any form of choice architecture that alters people's behaviour in a predictable way without restricting options or significantly changing their economic incentives. To count as a mere nudge, the intervention must require minimal intervention and must be cheap. Thaler and Sunstein wrote the book on nudges, quite literally. So their definition counts, and it's the one from…

In the case of organ donation, Thaler has written that he actually prefers mandated choice--i.e. there is no default but you have to either opt in or opt out--in this case. [1] But I'm not sure why a system where the government creates a default of either opt-in or opt-out (which you can change) wouldn't be a nudge. [1] https://www.nytimes.com/2009/09/27/business/economy/27view.h... (The issue being that in countries…

[deleted]

Re: No evidence for nudging after adjusting for publication bias

#80

Post by civilized »

pessimizer wrote: post #72
civilized wrote: post #51

Earlier quoted context omitted.

There's a lot of junk behavioral science out there, but things like "people often go with a default or recommended option so they can move on with their day" seem so obvious to me that I become suspicious of this debunking for debunking too much.

That's just refusing to be convinced by evidence, though. It's good to have hunches, but it's good to let them go after you've done the experiments. Come up with a new hunch and a new experiment that shows why the expected effects weren't seen, and you're right back in there.

"Refusing to be convinced by evidence" is a simplistic false dichotomy. The evidence is interesting but I have several reasons not to immediately take it as definitive.

Are you really certain that a big debunking in PNAS, surfing a wave of other celebrated debunkings, should be taken as definitive, when a good deal of the research being debunked was published to similar fanfare in PNAS back when a different kind of research was fashionable?

I take neither the original research nor the debunking as particularly credible. Without technical expertise, I'm left to educated guess. It's just my guess.

Post reply on HN