No evidence for nudging after adjusting for publication bias • Hacker News Board

Re: No evidence for nudging after adjusting for publication bias

#211

Post by unityByFreedom » Mon, Jul 25, 2022, 12:56 PM UTC

Schroedingersat wrote: post #201

unityByFreedom wrote: post #193

Earlier quoted context omitted.

> It brings “science” into disrepute to let social science associate with the other sciences. So you just want it renamed to "social studies" or what? What is your proposal, that nobody research this topic, or that they be separated in journals etc? I doubt that will have much impact on whether it makes the news. If you want that to change, you may need to get yourself onto the board of a journal you care about. > Th…

> So you just want it renamed to "social studies" or what? What is your proposal, that nobody research this topic, or that they be separated in journals etc? I doubt that will have much impact on whether it makes the news. Or maybe people just want to shine light on the fact that social science is harder than other science for a bunch of different reasons, bring social scientists' attention towards the tools that hel…

> maybe people just want to shine light on the fact that social science is harder than other science

The comment to which I was replying lacked the nuance in yours. Context is everything.

Re: No evidence for nudging after adjusting for publication bias

#212

Post by rayiner » Mon, Jul 25, 2022, 1:17 PM UTC

tptacek wrote: post #182

rayiner wrote: post #179

Earlier quoted context omitted.

Possibly. I recall nudges being a major talking point during the appointment: https://www.politico.com/agenda/story/2015/10/obamas-effort-... .

Your comment makes it seem as if the Nudge work had qualified him for OIRA, when in fact he was probably one of the country's most obvious lay-up candidates for that role. As I think you know, given his own background, it would have been weirder if Obama hadn't found a role in the government for Sunstein.

He released a book called “Nudge” the year before his appointment. I’m not saying he wasn’t qualified for OIRA without it but “making society better through subconscious manipulation” was definitely pitched as part of his shtick at the time.

Re: No evidence for nudging after adjusting for publication bias

#213

Post by concordDance » Mon, Jul 25, 2022, 1:22 PM UTC

rad88 wrote: post #191

concordDance wrote: post #160

Earlier quoted context omitted.

The default influenced me and pretty much everyone in my company Ive talked to (namely, almost everyone stuck to it). So yes, it defies credulity.

I believe you. If you know it influenced you though, that means you didn't ignore it or not even realize you could change it, which is the idea I was replying to. Again, it would be surprising if it didn't matter at all, but not unimaginable. What you're saying is that almost everybody in your company would have contributed a lesser amount if not for the default. It means you can all afford to give up $20k or whateve…

But they DO work in economics. Otherwise why would almost everyone contribute the default rather than a lower or higher amount?

Defaults are very strong when there is a lot of uncertainty about the payoffs of different answers.

Re: No evidence for nudging after adjusting for publication bias

#214

Post by harry8 » Mon, Jul 25, 2022, 1:36 PM UTC

TameAntelope wrote: post #184

harry8 wrote: post #177

Earlier quoted context omitted.

>> I don't think it matters if this criticism is right or not. > Okay, you and I care about very different things. Clearly it is being suggested that it doesn't matter with respect to the social studies departments being in dire need of drastic reform. If you don't care about that either direction why are you commenting on this thread? You do actually care one way and you're "point scoring" to further the argument? S…

Hm? I just think it matters if the letter (the original submission) is correct or not.

So are social studies departments and funding in dire need of reform in your opinion? How does the "correctness" in your view of the original submission affect that need or non-need?

Re: No evidence for nudging after adjusting for publication bias

#215

Post by salawat » Mon, Jul 25, 2022, 1:53 PM UTC

pizzathyme wrote: post #167

TameAntelope wrote: post #148

Earlier quoted context omitted.

Isn't this just one specific analysis using a very narrow definition of "nudge", one that doesn't even begin to encompass the work being done at those "300 "research" departments"? Further, isn't this using the same data from the original meta analysis that did find "of small to medium size" effect[0]? Why would this, alone, undo decades of research and clear, bright-line conclusions such as the ones cited in my sibl…

I agree, I have worked at 5 separate tech companies and have conducted hundreds of statistically significant experiments, changing what people select by default. These methods have been effective at helping hundreds of millions of people improve choices. It’s common practice, think: pricing on Amazon, default tips on Uber, default purchase price in a video game, etc. I don’t see how this would make me reinterpret all…

All you've proven is that controlling the default is a viable means of making a decision for someone.

I assure you, Opt-outs are much less likely to happen when combined with not explaining there is a choice to be made in the first place.

Re: No evidence for nudging after adjusting for publication bias

#216

Post by salawat » Mon, Jul 25, 2022, 1:57 PM UTC

jjgreen wrote: post #210

inglor_cz wrote: post #102

Earlier quoted context omitted.

Defaults very clearly work in matters such as consent to organ donation. In countries where you need to opt out of organ donation, few people bother to do so. Another question is whether this increases the total amount of successful donations. I was looking around for studies and found this one [1], which basically says "in some countries, yes". [1] https://behavioralpolicy.org/wp-content/uploads/2020/01/Does...

Without consent, it's not a donation. "Harvesting" would be an appropriate name.

Consider that 401k's are an onramp through which one delegates one's capital to be allocated by someone else, and it should be obvious the objective.

Capital wants to have the spigot left on. If people don't feed the beast voluntarily, Capital will make that the default.

Re: No evidence for nudging after adjusting for publication bias

#217

Post by curiousllama » Mon, Jul 25, 2022, 2:08 PM UTC

a_simm wrote: post #121

curiousllama wrote: post #117

Earlier quoted context omitted.

> This meta-analytic finding turns on the authors’ method for measuring publication bias. Because I accept that, I must believe that this entire body of research, probably the signal behavioral economics work, is essentially worthless! I strongly disagree with this statement, even as someone who believes “nudge” effects are wildly overblown. It means “these studies failed to find evidence” - NOT that there is nothing…

I defend and elucidate. Worthlessness I would define here relative to the amounts of public and of policy attention the nudge findings have received vs net value add modified by these results. Perhaps due to the PR efforts of leading researchers, it was much more than “set defaults intelligently.” The interpretations were more like: we can use social science to shape peoples’ behavior at the margins. Further these ma…

> statistically and therefore substantively insignificant

This is not what statistical significance implies. This misunderstanding, and its inverse, leads to the very errors for which you criticize the "nudge" papers.

More to the broader point, "set defaults intelligently" in fact implies the ability to "shape peoples' behavior at the margins." Otherwise, why bother thinking about them?

That's why what is actually at issue with "nudges" is effect size & context: how much of a difference can we have, and where?

And to that question, this paper provides little insight. It aggregates too much & ignores real-world policy evidence.

Now, it's still a good paper - people have gone WAY overboard with nudges in silly places - it just needs to be understood as "let's reign in expectations" and not "this field is bunk"

Re: No evidence for nudging after adjusting for publication bias

#218

Post by salawat » Mon, Jul 25, 2022, 2:11 PM UTC

t_mann wrote: post #139

a_simm wrote: post #37

Wow. As a former social scientist with an axe to grind this hits hard. I like to provide that HN community with some context as to what this means. There are some 300 “research” departments in each of the major social sciences: psychology, sociology, economics and anthropology. If you believe what they say, about half of their mission is teaching and the other half is research. That’s a lot, tens of billions of dolla…

Yes, nudging is an extremely well established concept, all the way from theory to policy - there's a Nobel (memorial) prize for the theory, and the UK government explicitly established a 'nudge unit' (the Behavioural Insights Team) to turn it into policy.

Manipulation and deceit works? And the Government is all in on it? Imagine that.

Re: No evidence for nudging after adjusting for publication bias

#219

Post by dx034 » Mon, Jul 25, 2022, 2:46 PM UTC

runarberg wrote: post #134

curiousllama wrote: post #117

Earlier quoted context omitted.

> This meta-analytic finding turns on the authors’ method for measuring publication bias. Because I accept that, I must believe that this entire body of research, probably the signal behavioral economics work, is essentially worthless! I strongly disagree with this statement, even as someone who believes “nudge” effects are wildly overblown. It means “these studies failed to find evidence” - NOT that there is nothing…

Can these effects be explained without inventing a new term? Because if they can then these studies didn’t really find anything did they? Whenever I see a new term being introduced as an explanation I am hesitant to accept it, as it may turn out to just be explaining the planetary motions with epicycle, when the motions can be easier explained by moving the sun to the center of the solar system instead of the earth.

Not very scientific, but isn't it just laziness? Most people (including me) are too lazy to think about all the choices they could make, so they just stay with the default choice most of the time. Not because they actually prefer it, mainly because they never even read it.

Re: No evidence for nudging after adjusting for publication bias

#220

Post by TameAntelope » Mon, Jul 25, 2022, 3:18 PM UTC

harry8 wrote: post #214

TameAntelope wrote: post #184

Earlier quoted context omitted.

Hm? I just think it matters if the letter (the original submission) is correct or not.

So are social studies departments and funding in dire need of reform in your opinion? How does the "correctness" in your view of the original submission affect that need or non-need?

Not sure I know enough to have an opinion about that; what I've been commenting about is the "nudge is dead" attitude that I don't think is warranted.