Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Is Ego-Depletion a Replicable Effect?

replicationindex.wordpress.com

Post reply on HN
Print view View original

41–50 of 53 posts

Re: Is Ego-Depletion a Replicable Effect?

#41

Post by onetwotree »

This really strikes me more as a problem of incentives than "lol psych researches need to learn statistics".

Incentives in this situation are funding and publication. These both require positive results. This seems to be tied to increasing monetization of science driving it away from a principled search for the truth.

We also see this with patents. My girlfriend has her first lab job this summer, and they had a day long seminar with the university's patent foundation on methods to ensure that nobody stole their ideas. This leads to a lot of secrecy (for example, this particular patent office doesn't allow researchers to store anything in the cloud, which speaks to lack of technical understanding, but also to an intense paranoia). This also dramatically reduces the capacity for sharing basic lab know how - there's nothing remotely like StackOverflow for scientists.

This issue is a bit near to my heart, as the company I work for was originally founded to help researchers share data in a secure way. It turned out that there wasn't any demand for that, because, as our CEO put it, "Everybody thinks their going to win the damn Nobel Prize". So we pivoted and now sell mostly to financial and tech companies that are serious about protecting user data.

Re: Is Ego-Depletion a Replicable Effect?

#42

Post by mysterypie »

The main thing I got out of the article was that I should ignore results from the fields of psychology or sociology for the foreseeable future. The standard of proof seems very low.

I never fell for Tony Robbins-type pop psychology, but I loved books like "Influence: The Psychology of Persuasion" by Robert Cialdini and "The Tipping Point" by Malcolm Gladwell.

I thought I was getting useful, non-obvious, actionable information. I'm disappointed that the lessons from those books might be BS as well.

Re: Is Ego-Depletion a Replicable Effect?

#43

Post by fdsaaf »

mysterypie wrote: post #42

The main thing I got out of the article was that I should ignore results from the fields of psychology or sociology for the foreseeable future. The standard of proof seems very low. I never fell for Tony Robbins-type pop psychology, but I loved books like "Influence: The Psychology of Persuasion" by Robert Cialdini and "The Tipping Point" by Malcolm Gladwell. I thought I was getting useful, non-obvious, actionable in…

> I should ignore results from the fields of psychology or sociology for the foreseeable future

Yes, you should. The problem is that some of these bad results have become key political tenets. Try voicing skepticism about the reality of stereotype threat and see how long it is before HR finds some excuse to fire you.

If you haven't noticed, there are powerful social pressures in the Bay Area to go along with certain kinds of reality denial. The more reality suggests that dominant Bay Area social beliefs are false, the louder supporters reiterate these beliefs and the greater the ferocity with which they punish dissent.

Re: Is Ego-Depletion a Replicable Effect?

#44

Post by mysterypie »

fdsaaf wrote: post #43
mysterypie wrote: post #42

The main thing I got out of the article was that I should ignore results from the fields of psychology or sociology for the foreseeable future. The standard of proof seems very low. I never fell for Tony Robbins-type pop psychology, but I loved books like "Influence: The Psychology of Persuasion" by Robert Cialdini and "The Tipping Point" by Malcolm Gladwell. I thought I was getting useful, non-obvious, actionable in…

> I should ignore results from the fields of psychology or sociology for the foreseeable future Yes, you should. The problem is that some of these bad results have become key political tenets. Try voicing skepticism about the reality of stereotype threat and see how long it is before HR finds some excuse to fire you. If you haven't noticed, there are powerful social pressures in the Bay Area to go along with certain…

I didn't know the term "stereotype threat" and had to google it. Here is one of the clearer explanations I found:

"Black and White college students a half-hour test using difficult items from the verbal Graduate Record Exam (GRE). In the stereotype-threat condition, they told students the test diagnosed intellectual ability, thus potentially eliciting the stereotype that Blacks are less intelligent than Whites. In the no-stereotype-threat condition, the researchers told students that the test was a problem-solving lab task that said nothing about ability, presumably rendering stereotypes irrelevant. In the stereotype threat condition, Blacks did less well than Whites. In the no stereotype-threat condition, Blacks' performance rose to match that of equally skilled Whites."[1]

"Merely telling women that a math test does not show gender differences improved their test performance. The researchers gave a math test to men and women after telling half the women that the test had shown gender differences, and telling the rest that it found none. When test administrators told women that that tests showed no gender differences, the women performed equal to men. Those who were told the test showed gender differences did significantly worse than men, just like women who were told nothing about the test."[1]

[1] http://www.apa.org/research/action/stereotype.aspx

(I'm not saying that this effect is real. I'm just passing along an explanation of the term for the benefit of others.)

Re: Is Ego-Depletion a Replicable Effect?

#45

Post by popra »

fdsaaf wrote: post #43
mysterypie wrote: post #42

The main thing I got out of the article was that I should ignore results from the fields of psychology or sociology for the foreseeable future. The standard of proof seems very low. I never fell for Tony Robbins-type pop psychology, but I loved books like "Influence: The Psychology of Persuasion" by Robert Cialdini and "The Tipping Point" by Malcolm Gladwell. I thought I was getting useful, non-obvious, actionable in…

> I should ignore results from the fields of psychology or sociology for the foreseeable future Yes, you should. The problem is that some of these bad results have become key political tenets. Try voicing skepticism about the reality of stereotype threat and see how long it is before HR finds some excuse to fire you. If you haven't noticed, there are powerful social pressures in the Bay Area to go along with certain…

> HR finds some excuse to fire you

Which shouldn't be surprising, because unless carefully expressed, skepticism towards the stereotype threat effect can be interpreted as a justification for stereotyping, especially when the skepticism is expressed in the context of political tenets.

Re: Is Ego-Depletion a Replicable Effect?

#46

Post by pknight »

TeMPOraL wrote: post #23
pknight wrote: post #17

Earlier quoted context omitted.

If your view is a priori that psi effects are impossible all you will see in the studies that 'it must be invalid and flawed or faked'. Yes, this makes people doubt scientific practices in general. You will always feel you are winning that argument, no matter what criteria those scientists meet. But psi researchers these days jump through far more hoops, run far more replications, spend far more time looking for weak…

> But psi researchers these days jump through far more hoops, run far more replications, spend far more time looking for weaknesses and alternative explanations, garner far more scrutiny and second guessing. To call that a 'fake field of research' when it is more rigorous is disingenuous. But that's the point of Scott's article. We have more than enough reasons to believe the whole field is bullshit (did you see a wo…

Explaining away thousands of studies that happen to study controversial effects with unknown mechanisms in one brush is not demonstrating intellectual rigour nor does it make you come across as upholding 'real science'.

The explanation that psi-researchers are just good at finding the appearance of signal from noise, small as it may be, is fine, but then personally, I'd like to see the same vigour for explaining what the signal actually is. And people that have a firm belief that any 'psi'-explanation can't possibly true (and this a pretty arbitrary notion at that, as psi would just follow conventional physics and adhere to normal biological mechanisms) just aren't going to accept any other explanation. So they look for issues with the statistics used.

The notition that the whole field is BS, as you put it, requires all the researchers in involved to be utterly mad, stupid or deceitful. That is a claim easily made by skeptics, a thing that they have no trouble agreeing to. That view is not born out of scientific honesty, it's just another form of cargo-cult thinking.

Re: Is Ego-Depletion a Replicable Effect?

#47

Post by pknight »

Dylan16807 wrote: post #21
pknight wrote: post #17

Earlier quoted context omitted.

If your view is a priori that psi effects are impossible all you will see in the studies that 'it must be invalid and flawed or faked'. Yes, this makes people doubt scientific practices in general. You will always feel you are winning that argument, no matter what criteria those scientists meet. But psi researchers these days jump through far more hoops, run far more replications, spend far more time looking for weak…

> But psi researchers these days jump through far more hoops, run far more replications, spend far more time looking for weaknesses and alternative explanations, garner far more scrutiny and second guessing. To call that a 'fake field of research' when it is more rigorous is disingenuous. So what's a good study you can point me at? The nice thing about psi testing is that you can get a result sample every few seconds…

Try: http://www.ingentaconnect.com/content/pe/pe/2016/00000029/00...

"During 2013 and 2014, a total of 1479 people from 77 countries contributed 2985 test sessions. Over the same period, 5738 sessions were run as controls by a computer programmed to simulate human participants. The results showed that with human observers the fringe visibility at the center of the interference pattern deviated from a null effect by 5.72 sigma (p = 1.05 × 10−8), with the direction of the deviation conforming to the observers' intentions. The same analysis applied to the control data resulted in an overall deviation of −0.17 sigma."

There are some presentations on Youtube that go into the details of the double-slit line of experiments.

Re: Is Ego-Depletion a Replicable Effect?

#48

Post by pknight »

tgb wrote: post #22
pknight wrote: post #17

Earlier quoted context omitted.

If your view is a priori that psi effects are impossible all you will see in the studies that 'it must be invalid and flawed or faked'. Yes, this makes people doubt scientific practices in general. You will always feel you are winning that argument, no matter what criteria those scientists meet. But psi researchers these days jump through far more hoops, run far more replications, spend far more time looking for weak…

The point of the linked article is exactly that psi researchers go through far more rigorous methods than most researchers. The fact that they still come up with wrong results tells us just how far the goal posts need to be moved - but not just for psi, for all of psychology and other fields too.

This hinges on the premise that any kind of psi-like-effect is impossible. And that comes from a belief rather than from evidence.

Re: Is Ego-Depletion a Replicable Effect?

#49

Post by fdsaaf »

popra wrote: post #45
fdsaaf wrote: post #43

Earlier quoted context omitted.

> I should ignore results from the fields of psychology or sociology for the foreseeable future Yes, you should. The problem is that some of these bad results have become key political tenets. Try voicing skepticism about the reality of stereotype threat and see how long it is before HR finds some excuse to fire you. If you haven't noticed, there are powerful social pressures in the Bay Area to go along with certain…

> HR finds some excuse to fire you Which shouldn't be surprising, because unless carefully expressed, skepticism towards the stereotype threat effect can be interpreted as a justification for stereotyping, especially when the skepticism is expressed in the context of political tenets.

Stereotypes are accurate, though. People are good pattern-recognizers, and the characteristics that we use to form stereotype do convey useful information.

Pretending that stereotypes are factually bogus is wrong. It creates resentment. It's much better to accept that stereotypes exist for a reason, but emphasize that it's a grave injustice to let stereotypes hurt our evaluation of people who defy these stereotypes.

> skepticism towards the stereotype threat effect can be interpreted as a justification for stereotyping

Current HR policies go beyond requiring fair treatment of individuals as individuals. Every sane person supports doing that. HR demands that we silence truths about the natural world go unstated. HR demands that we make untrue statements about the world. That's anathema to anyone technically-minded.

Re: Is Ego-Depletion a Replicable Effect?

#50

Post by dwaltrip »

fao_ wrote: post #30
TeMPOraL wrote: post #20

Earlier quoted context omitted.

This should be publicly recorded too and maybe eventually we'll have a list of funding bodies that like to minmax profits by damaging research.

This doesn't seem very feasible given the current level of privacy afforded to companies. I don't think there's a way to distinguish between studies that have lost funding because there are genuinely less funds to go around, and foo experiment just happened to be one of the ones that didn't make the cut, and studies that have had the funding pulled for harmful reasons (Like negative results). Mainly because it's devi…

What about starting by implementing these measures in public universities? That seems very feasible, and could have a snowball effect on how studies are broadly analyzed and perceived.
Post reply on HN