Is Ego-Depletion a Replicable Effect?
replicationindex.wordpress.com
Is Ego-Depletion a Replicable Effect?
1–10 of 53 posts
Re: Is Ego-Depletion a Replicable Effect?
#2As a layman when it comes to statistics and academia, this is the first I've heard of "publication bias". It seems like a pretty big problem at first glance, especially if it recurses and calls into question meta-analysis being done to investigate the presence of the bias itself. Sounds like a massive headache.
If this problem is as big as it sounds, what are the repercussions?
Re: Is Ego-Depletion a Replicable Effect?
#3As a layman when it comes to statistics and academia, this is the first I've heard of "publication bias". It seems like a pretty big problem at first glance, especially if it recurses and calls into question meta-analysis being done to investigate the presence of the bias itself. Sounds like a massive headache. If this problem is as big as it sounds, what are the repercussions?
Think about it this way: Suppose my theory is that a fair coin is not actually fair but always lands on heads.
If I flip a coin one million times and only publish the ~500,000 results that show coin toss coming up as "heads," I could lead people to believe that a fair coin will always land on heads.
That's how misleading this bias can be.
But anyone can toss a fair coin, so I'd be found out pretty quick. But with more complicated and murky stuff like psychology this can be a huge problem, especially if people begin using this shaky work as foundation for things like: public policy, how to live their lives, how to do work, etc.
After all they can't just toss a coin to check my work on the more complicated stuff.
Re: Is Ego-Depletion a Replicable Effect?
#4I'm glad to see sociology is becoming more rigorous.
Re: Is Ego-Depletion a Replicable Effect?
#5As a layman when it comes to statistics and academia, this is the first I've heard of "publication bias". It seems like a pretty big problem at first glance, especially if it recurses and calls into question meta-analysis being done to investigate the presence of the bias itself. Sounds like a massive headache. If this problem is as big as it sounds, what are the repercussions?
It's a rather old subject, and much richer than just publication bias, so there's plenty written about it. There is a good introduction on Andrew Gelman's blog, for example. http://andrewgelman.com/2016/03/05/29195/ http://andrewgelman.com/category/zombies/
In particular, publication bias isn't quite as bad as the garden-of-forking-paths issue.
Re: Is Ego-Depletion a Replicable Effect?
#6As a layman when it comes to statistics and academia, this is the first I've heard of "publication bias". It seems like a pretty big problem at first glance, especially if it recurses and calls into question meta-analysis being done to investigate the presence of the bias itself. Sounds like a massive headache. If this problem is as big as it sounds, what are the repercussions?
Think about it this way: Suppose my theory is that a fair coin is not actually fair but always lands on heads. If I flip a coin one million times and only publish the ~500,000 results that show coin toss coming up as "heads," I could lead people to believe that a fair coin will always land on heads. That's how misleading this bias can be. But anyone can toss a fair coin, so I'd be found out pretty quick. But with mor…
I think what you're describing is just fraud.
To take your fair coin example, publication bias is when the only studies that get published are the ones that show an unfairness in the coin – the rest of the studies that simply find 50% heads and 50% tails (aka a fair coin) don't get published because they're not interesting.
Re: Is Ego-Depletion a Replicable Effect?
#7Ego depletion is framed as an effect in a particular direction: applying self-control reduces self-control in the near future. So studies are called successful if they see that direction, otherwise unsuccessful.
But the other direction is a reasonable hypothesis too: applying self-control increases self-control because the brain regions are activated and ready, or something.
So you could frame the problem bidirectionally: applying self-control affects self-control ability according to some time-dependent formula with parameters that we'd like to estimate.
Framing the problem bidirectionally might reduce publication bias, since every estimate of the parameters would seem equally valuable.
Re: Is Ego-Depletion a Replicable Effect?
#8As a layman when it comes to statistics and academia, this is the first I've heard of "publication bias". It seems like a pretty big problem at first glance, especially if it recurses and calls into question meta-analysis being done to investigate the presence of the bias itself. Sounds like a massive headache. If this problem is as big as it sounds, what are the repercussions?
Yes it's a very big problem. One of my favorite meta analysis tools is used to detect and correct for it.
Re: Is Ego-Depletion a Replicable Effect?
#9As a layman when it comes to statistics and academia, this is the first I've heard of "publication bias". It seems like a pretty big problem at first glance, especially if it recurses and calls into question meta-analysis being done to investigate the presence of the bias itself. Sounds like a massive headache. If this problem is as big as it sounds, what are the repercussions?
Oh it's far worse than that. Iirc about a third of studies fail to replicate, and in psychology it might be higher. In fact we even have a control group for science. A fake field of research that spends it's time studying phenomena which can't possibly be real, the paranormal researchers. They produce tons of positive results anyway. See The Control Group is Out of Control: http://slatestarcodex.com/2014/04/28/the-control-group-is-ou...
Re: Is Ego-Depletion a Replicable Effect?
#10As a layman when it comes to statistics and academia, this is the first I've heard of "publication bias". It seems like a pretty big problem at first glance, especially if it recurses and calls into question meta-analysis being done to investigate the presence of the bias itself. Sounds like a massive headache. If this problem is as big as it sounds, what are the repercussions?
The repercussions are pretty depressing for social sciences. Especially Psych, which relies on a lot of "30 undergraduates were recruited" to demonstrate effects. Particularly damaging is that this particular theory (thinking hard leads to drops in blood glucose levels) was widely promoted in the general public; I recall watching a TED talk about it, as well as a pop-sci book.
There was an econtalk episode late last year with Brian Nosek about the Reproducability Project (http://www.econtalk.org/archives/2015/11/brian_nosek_on.html). They went the further step of analyzing the wider field to see how bad the biases are. Previous attempts to call out replication (priming theory) did not go well, leading to a lot of hostility. Nosek's process is a bit more friendly, by inviting the original authors's critique early, before data collection. I figure it's a bit harder to blast the auditors for considering a replication study without appearing as though you doubt the effect is real yourself. They still got a bit of pushback, but nowhere near as acrimonious.