Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

No evidence for nudging after adjusting for publication bias

pnas.org

Post reply on HN
Print view View original

51–60 of 246 posts

Re: No evidence for nudging after adjusting for publication bias

#51

Post by civilized »

pessimizer wrote: post #42
themitigating wrote: post #24

Earlier quoted context omitted.

Isn't that always the case?

I think some people careen from fully trusting one thing to fully trusting the opposite thing. If you're not one of those people, you'll never understand dismissing things on the basis of "you should be critical of this, because not everything that people say is true." I do feel like that, even though being critical is something we should always do, that in cases where 1) the only reason you started paying attention…

There's a lot of junk behavioral science out there, but things like "people often go with a default or recommended option so they can move on with their day" seem so obvious to me that I become suspicious of this debunking for debunking too much.

Re: No evidence for nudging after adjusting for publication bias

#52

Post by dr_dshiv »

akyu wrote: post #11

No evidence for nudging =/= nudging doesn't exist. I'm fairly sure anyone who has done A/B testing at scale has plenty of evidence that nudging works. Perhaps not up to the standard of science, but there are literally people who manipulate choice architecture for a living and I'm fairly convinced a lot of that stuff actually works.

Seriously, what about that kind of publication bias: A/B tests don’t get published.

If you run a useful system where it would be meaningful and interesting to know whether a social science theory actually applied, you might run an A/B test to see if it works. If it works, it is adopted—but it is almost never published. And that is for two reasons: 1. no incentive to publish and 2. major incentive not to publish. #2 is recent (post Facebook experiment) and it is specifically because a large portion of the educated public accepts invisible A/B testing but recoils with moral indignation at the use of A/B testing results in published science. Too bad: Facebook keeps testing social science theories, but no longer publishes the results.

Re: No evidence for nudging after adjusting for publication bias

#53

Post by aaaaaaaaaaab »

lIl-IIIl wrote: post #20
omginternets wrote: post #13

Earlier quoted context omitted.

What exactly makes you convinced that it works? To be specific: why wouldn’t there be bias in the A/B testing results, too? There are literally people who give astrological analyses for a living.

We are talking about publication bias, where the decision whether to publish something is biased by the outcome of the experiment. I think this doesn't really apply to A/B testing, because people are incentivized pay as much attention to negative results as to positive ones.

People are incentivized to pay attention to the result that increases their mid-year bonus the most.

Re: No evidence for nudging after adjusting for publication bias

#54

Post by nabla9 »

civilized wrote: post #44
nabla9 wrote: post #38

Earlier quoted context omitted.

There is no evidence of impact. That's different from the existence of the phenomenon. Same thing happened to Kahneman, Daniel (2011) and his book of Thinking, Fast and Slow. He acknowledges that several pieces of evidence he presents in the book has disappeared and can't be replicated. He still thinks he is right, he just admits that he does not have strong evidence anymore. What is left is a theory with less and le…

I just think it's implausible that there's been no good research showing solid evidence of choice architecture mattering. Could be, I'm not an expert, but I'd like to see where things stand after a couple more years of research and debate.

>I just think it's implausible that there's b

The job of scientist is to find and present that evidence.

If your assumption is correct, and Kahneman failed to find good evidence that was there, that makes him a incompetent scientist. I don't think he is.

Re: No evidence for nudging after adjusting for publication bias

#57

Post by JusticeJuice »

xyzzy123 wrote: post #46
a_simm wrote: post #37

Wow. As a former social scientist with an axe to grind this hits hard. I like to provide that HN community with some context as to what this means. There are some 300 “research” departments in each of the major social sciences: psychology, sociology, economics and anthropology. If you believe what they say, about half of their mission is teaching and the other half is research. That’s a lot, tens of billions of dolla…

In the example you cited I don't think that's a nudge? Or is it? I ask because I am sure that changing defaults DEFINITELY works, especially if the user does not have a strong existing preference. You're not really changing user behaviour most of the time, you're changing the outcome of what they're trying to do, which is to reduce their cognitive load by ignoring as much as they possibly can.

Nudges are often imagined as just how choices are presented, but yes the default option is considered part of nudge theory. As also is social proof ("Your friends picked this choice").

https://en.wikipedia.org/wiki/Nudge_theory#Types_of_nudges

Re: No evidence for nudging after adjusting for publication bias

#58

Post by mikkergp »

xyzzy123 wrote: post #46
a_simm wrote: post #37

Wow. As a former social scientist with an axe to grind this hits hard. I like to provide that HN community with some context as to what this means. There are some 300 “research” departments in each of the major social sciences: psychology, sociology, economics and anthropology. If you believe what they say, about half of their mission is teaching and the other half is research. That’s a lot, tens of billions of dolla…

In the example you cited I don't think that's a nudge? Or is it? I ask because I am sure that changing defaults DEFINITELY works, especially if the user does not have a strong existing preference. You're not really changing user behaviour most of the time, you're changing the outcome of what they're trying to do, which is to reduce their cognitive load by ignoring as much as they possibly can.

Some other poster posted that they must have a pretty specific definition of nudge, because it defies credulity that defaults don’t change outcome if only because half the time I don’t read the defaults or know where to find them.

I mean I just found out two weeks ago you could change the hacker news banner color. Are you telling me I’m in a statistically insignia can’t minority of hacker news users?

Also how many settings are there in the average application, you can’t tell me most users go through all of those settings to get exactly what they want.

Re: No evidence for nudging after adjusting for publication bias

#60

Post by xyzzy123 »

mikkergp wrote: post #58
xyzzy123 wrote: post #46

Earlier quoted context omitted.

In the example you cited I don't think that's a nudge? Or is it? I ask because I am sure that changing defaults DEFINITELY works, especially if the user does not have a strong existing preference. You're not really changing user behaviour most of the time, you're changing the outcome of what they're trying to do, which is to reduce their cognitive load by ignoring as much as they possibly can.

Some other poster posted that they must have a pretty specific definition of nudge, because it defies credulity that defaults don’t change outcome if only because half the time I don’t read the defaults or know where to find them. I mean I just found out two weeks ago you could change the hacker news banner color. Are you telling me I’m in a statistically insignia can’t minority of hacker news users? Also how many se…

If defaults don't work then Google wasted 15 billion dollars last year paying Apple to be in the search bar...

I guess there must be further detail in the paper and I will have to read it to understand the nuance.

Post reply on HN