Is Ego-Depletion a Replicable Effect? • Hacker News Board

Re: Is Ego-Depletion a Replicable Effect?

#21

Post by Dylan16807 » Mon, Jul 11, 2016, 10:38 AM UTC

pknight wrote: post #17

Houshalter wrote: post #9

Earlier quoted context omitted.

Oh it's far worse than that. Iirc about a third of studies fail to replicate, and in psychology it might be higher. In fact we even have a control group for science. A fake field of research that spends it's time studying phenomena which can't possibly be real, the paranormal researchers. They produce tons of positive results anyway. See The Control Group is Out of Control : http://slatestarcodex.com/2014/04/28/the-c…

If your view is a priori that psi effects are impossible all you will see in the studies that 'it must be invalid and flawed or faked'. Yes, this makes people doubt scientific practices in general. You will always feel you are winning that argument, no matter what criteria those scientists meet. But psi researchers these days jump through far more hoops, run far more replications, spend far more time looking for weak…

> But psi researchers these days jump through far more hoops, run far more replications, spend far more time looking for weaknesses and alternative explanations, garner far more scrutiny and second guessing. To call that a 'fake field of research' when it is more rigorous is disingenuous.

So what's a good study you can point me at? The nice thing about psi testing is that you can get a result sample every few seconds. It should be easy to turn even a moderately small effect into a five sigma result that's immune to publication bias.

Re: Is Ego-Depletion a Replicable Effect?

#22

Post by tgb » Mon, Jul 11, 2016, 10:39 AM UTC

pknight wrote: post #17

Houshalter wrote: post #9

Earlier quoted context omitted.

Oh it's far worse than that. Iirc about a third of studies fail to replicate, and in psychology it might be higher. In fact we even have a control group for science. A fake field of research that spends it's time studying phenomena which can't possibly be real, the paranormal researchers. They produce tons of positive results anyway. See The Control Group is Out of Control : http://slatestarcodex.com/2014/04/28/the-c…

If your view is a priori that psi effects are impossible all you will see in the studies that 'it must be invalid and flawed or faked'. Yes, this makes people doubt scientific practices in general. You will always feel you are winning that argument, no matter what criteria those scientists meet. But psi researchers these days jump through far more hoops, run far more replications, spend far more time looking for weak…

The point of the linked article is exactly that psi researchers go through far more rigorous methods than most researchers. The fact that they still come up with wrong results tells us just how far the goal posts need to be moved - but not just for psi, for all of psychology and other fields too.

Re: Is Ego-Depletion a Replicable Effect?

#23

Post by TeMPOraL » Mon, Jul 11, 2016, 10:42 AM UTC

pknight wrote: post #17

Houshalter wrote: post #9

Earlier quoted context omitted.

Oh it's far worse than that. Iirc about a third of studies fail to replicate, and in psychology it might be higher. In fact we even have a control group for science. A fake field of research that spends it's time studying phenomena which can't possibly be real, the paranormal researchers. They produce tons of positive results anyway. See The Control Group is Out of Control : http://slatestarcodex.com/2014/04/28/the-c…

If your view is a priori that psi effects are impossible all you will see in the studies that 'it must be invalid and flawed or faked'. Yes, this makes people doubt scientific practices in general. You will always feel you are winning that argument, no matter what criteria those scientists meet. But psi researchers these days jump through far more hoops, run far more replications, spend far more time looking for weak…

> But psi researchers these days jump through far more hoops, run far more replications, spend far more time looking for weaknesses and alternative explanations, garner far more scrutiny and second guessing. To call that a 'fake field of research' when it is more rigorous is disingenuous.

But that's the point of Scott's article. We have more than enough reasons to believe the whole field is bullshit (did you see a working practical application of that research?), but because it does all the moves and rituals of Real Science, it forms a perfect baseline for how many positive results we can expect with pure cargo-cult science.

Re: Is Ego-Depletion a Replicable Effect?

#24

Post by TeMPOraL » Mon, Jul 11, 2016, 10:45 AM UTC

apatters wrote: post #16

So... do I eat the chocolate cookies, or not? This is so difficult!

Run an n=1 study on yourself. Even if it's full of placebo effects and doesn't generalize to anyone else, as long as you figured out what makes you feel/do better, you win.

Re: Is Ego-Depletion a Replicable Effect?

#25

Post by Cybiote » Mon, Jul 11, 2016, 10:56 AM UTC

tlb wrote: post #7

Ego depletion is framed as an effect in a particular direction: applying self-control reduces self-control in the near future. So studies are called successful if they see that direction, otherwise unsuccessful. But the other direction is a reasonable hypothesis too: applying self-control increases self-control because the brain regions are activated and ready, or something. So you could frame the problem bidirection…

The common mistake that many (but not necessarily you) make is to assume that conscious effort is proportional to energy use in the brain. The idea that willpower brain is some kind of store that can be depleted is not as parsimonious as other hypotheses.

Cognizing is a costly activity, whether daydreaming or thinking, a lot of stuff is going on. So why does thinking in particular feel difficult?

The least terrible analogy I can think of is to imagine a distribution network. Postal systems, railways, the internet; whether they are at 20% or 100% utilization, the marginal costs don't budge much. The other key concept is that the brain both learns as well as has to figure out how to do so efficiently. Reward schedules and surprises play prominently into this.

The hypothesis: some things feel difficult in a way not directly related to energy use because it's attention that is key and it is better thought of as a resource to be allocated and managed.

The questions are: why is it so hard to maintain focus on abstractions for extended periods of time? Someone is going to mention the ADHD hunter ancestor theory as selectors for extended focus, but even then, I will say that their attention still did not stay as fixed or devoid of obvious rewards as some abstract thinking requires. Is focus trainable? Why are some better at not responding to immediate rewards? Is this trainable?

Re: Is Ego-Depletion a Replicable Effect?

#26

Post by facepalm » Mon, Jul 11, 2016, 11:04 AM UTC

While the picture seems bad for Ego-Depletion, I find the counter arguments put forward in the article not wholly satisfying, either.

So they tried to replicate it in a large effort and failed, but they didn't use the same test as in the original paper. How is that a replication study? I think to properly fail a replication study, you should use the same test as in the original study. In the replication the used many participants, but developed a cheaper test to administer (wholly computer based).

As it is, they have another indicator that Ego Depletion might not be real, but just as well there could be some other effect that explains why the replication failed.

Likewise for meta analysis: it looks good on paper, how could an average of n studies be wrong? Surely the mass of papers studied will cancel out all the flaws of the individual studies? But on the other hand it seems to me meta analysis would overestimate the impact of bad studies, as they are easier to make so there a probably more of them.

A meta study is certainly an interesting data point, but I feel it can not always be the end of the story. It can maybe point out the need for deeper questions or further research, but not be proof on it's own.

Re: Is Ego-Depletion a Replicable Effect?

#27

Post by adrianratnapala » Mon, Jul 11, 2016, 12:00 PM UTC

pknight wrote: post #17

Houshalter wrote: post #9

Earlier quoted context omitted.

Oh it's far worse than that. Iirc about a third of studies fail to replicate, and in psychology it might be higher. In fact we even have a control group for science. A fake field of research that spends it's time studying phenomena which can't possibly be real, the paranormal researchers. They produce tons of positive results anyway. See The Control Group is Out of Control : http://slatestarcodex.com/2014/04/28/the-c…

If your view is a priori that psi effects are impossible all you will see in the studies that 'it must be invalid and flawed or faked'. Yes, this makes people doubt scientific practices in general. You will always feel you are winning that argument, no matter what criteria those scientists meet. But psi researchers these days jump through far more hoops, run far more replications, spend far more time looking for weak…

Just to clear something up, Haushalter did not claim physchology was a a "fake field of research" -- he was saying that paranormal research was fake.

In this way, paranomral reserach is a good "control group" for science because it uses methods similar to those of psychology and the social sciences.

Re: Is Ego-Depletion a Replicable Effect?

#28

Post by bhouston » Mon, Jul 11, 2016, 12:10 PM UTC

Cybiote wrote: post #25

tlb wrote: post #7

Ego depletion is framed as an effect in a particular direction: applying self-control reduces self-control in the near future. So studies are called successful if they see that direction, otherwise unsuccessful. But the other direction is a reasonable hypothesis too: applying self-control increases self-control because the brain regions are activated and ready, or something. So you could frame the problem bidirection…

The common mistake that many (but not necessarily you) make is to assume that conscious effort is proportional to energy use in the brain. The idea that willpower brain is some kind of store that can be depleted is not as parsimonious as other hypotheses. Cognizing is a costly activity, whether daydreaming or thinking, a lot of stuff is going on. So why does thinking in particular feel difficult? The least terrible a…

I understand focus is related to dopamine, a neurochemical and it is know that nearly all neurochemicals can be depleted or their pathways become less effective because of various run-time compensations that occur.

Coffee effects me most for the first cup, most likely because I have stores of adrenaline and dopamine, after than first cup any cup after than decreases its effect on me. The first coffee probably is more noticable because of the difference between my no coffee state and my coffee state -- which is difference than a wearing off coffee state and a free cup coffee state.

It is reasonable to be able to measure these effects. Not saying this effect is true, but neurochemical store depletion as well as real-time and long-term compensation are real.

Re: Is Ego-Depletion a Replicable Effect?

#29

Post by algorias » Mon, Jul 11, 2016, 12:45 PM UTC

cleeus wrote: post #13

love this quote: > Every researcher knows about publication bias, but the practice is so widespread that it is not considered a serious problem.

It is considered a serious problem by many, being a key part of the whole replication crisis. The idea that studies need to be preregistered is slowly gaining ground I feel.

Re: Is Ego-Depletion a Replicable Effect?

#30

Post by fao_ » Mon, Jul 11, 2016, 1:01 PM UTC

TeMPOraL wrote: post #20

fao_ wrote: post #19

Earlier quoted context omitted.

What if funding for a study gets withdrawn because it looks like it'll produce negative/unfavorable results?

This should be publicly recorded too and maybe eventually we'll have a list of funding bodies that like to minmax profits by damaging research.

This doesn't seem very feasible given the current level of privacy afforded to companies.

I don't think there's a way to distinguish between studies that have lost funding because there are genuinely less funds to go around, and foo experiment just happened to be one of the ones that didn't make the cut, and studies that have had the funding pulled for harmful reasons (Like negative results).

Mainly because it's devilishly easy to mask; you'd just reduce funding to research and redistribute it into something plausible like marketing. Of course this isn't feasible for lots of studies producing negative results.

Of course, one way to counter that is to insist that the data is published regardless, but I'm sure that the data could be hidden with clever use of NDAs and court arguments along the lines of "We need to prohibit the distribution of company assets".