> perceived uniqueness is indeed highly statistically significantly associated with underperformance
Isn't it possible that perceived unique projects are unique and therefore are harder and underperform? Did they control for that?
Uniqueness Bias: Why it matters, how to curb it
11–20 of 70 posts
Re: Uniqueness Bias: Why it matters, how to curb it
#12I read this book called "How Big Things Get Done." I've seen my fair share of projects going haywire and I wanted to understand if we could do better. The book identifies uniqueness bias as an important reason for why most big projects overrun. (*) In short, this bias leads planners to view their projects as unique, thereby disregarding valuable lessons from previous similar projects. (*) The book compiles 16,000 big…
This goes beyond project estimations. The Metaculus tournaments have questions like > Will YouTube be banned in Russia before October 1? These are the kinds of questions most people I meet would claim are impossible to forecast accurately on because of their supposed uniqueness, yet the Metaculus community does it again and again. I believe the problem is a lack of statistical literacy. Back in the 1500s shipping ins…
> the Metaculus community does it again and again.
Do they? I can't see where on that site there is something like "history of predictions" or "track record".
Edit: here.
https://www.metaculus.com/questions/track-record/
But now I can't tell if it's good or bad.
Re: Uniqueness Bias: Why it matters, how to curb it
#13Earlier quoted context omitted.
This goes beyond project estimations. The Metaculus tournaments have questions like > Will YouTube be banned in Russia before October 1? These are the kinds of questions most people I meet would claim are impossible to forecast accurately on because of their supposed uniqueness, yet the Metaculus community does it again and again. I believe the problem is a lack of statistical literacy. Back in the 1500s shipping ins…
I know you're coming from a good place, but the 'most people I meet don't understand this' line about statistics is quite arrogant. Most people you meet are fully capable of understanding statistics; you should do a better job of explaining it when it comes up, or maybe you are the one who misunderstands. After all, most statisticians thought Marilyn vos Savant was wrong about the goats too...
Not my experience at all. Just one example: try talking to physicians about false discovery rates; even those who do not profit from state-of-the art screening methods. Incorporating Bayesian methods is even a struggle for statisticians. It is a stuggle for me.
John Ioannidis has much to say on this topic:
Re: Uniqueness Bias: Why it matters, how to curb it
#14> perceived uniqueness is indeed highly statistically significantly associated with underperformance Isn't it possible that perceived unique projects are unique and therefore are harder and underperform? Did they control for that?
Or that people who don't know many scenarios of the past cannot draw upon similarities or see the connection without experience and thus conclude their situation is unique.
Re: Uniqueness Bias: Why it matters, how to curb it
#15I read this book called "How Big Things Get Done." I've seen my fair share of projects going haywire and I wanted to understand if we could do better. The book identifies uniqueness bias as an important reason for why most big projects overrun. (*) In short, this bias leads planners to view their projects as unique, thereby disregarding valuable lessons from previous similar projects. (*) The book compiles 16,000 big…
This goes beyond project estimations. The Metaculus tournaments have questions like > Will YouTube be banned in Russia before October 1? These are the kinds of questions most people I meet would claim are impossible to forecast accurately on because of their supposed uniqueness, yet the Metaculus community does it again and again. I believe the problem is a lack of statistical literacy. Back in the 1500s shipping ins…
> Back in the 1500s shipping insurance was priced under the assumption that if you just learned enough about the voyage you could tell for certain whether it would be successful or not.
Do you have a source on that? That sounds very unlikely to me. People just have to look at a single storm to see that it sometimes destroys some ships and does not destroy others. It is very clearly has a luck component. How could have anyone (at any time really) believe that "if you just learned enough about the voyage you could tell for certain whether it would be successful or not"?
Re: Uniqueness Bias: Why it matters, how to curb it
#16Perhaps a good way of thinking about this is: "history doesn't repeat itself, but it rhymes" - (supposedly) Mark Twain
Re: Uniqueness Bias: Why it matters, how to curb it
#17Earlier quoted context omitted.
This goes beyond project estimations. The Metaculus tournaments have questions like > Will YouTube be banned in Russia before October 1? These are the kinds of questions most people I meet would claim are impossible to forecast accurately on because of their supposed uniqueness, yet the Metaculus community does it again and again. I believe the problem is a lack of statistical literacy. Back in the 1500s shipping ins…
> the Metaculus community does it again and again. Do they? I can't see where on that site there is something like "history of predictions" or "track record". Edit: here. https://www.metaculus.com/questions/track-record/ But now I can't tell if it's good or bad.
The binary calibration diagram is what I would focus on. Of all the times Metaculus has said there's an x % of something happening, it has happened nearly exactly x % of the time. That is both useful and a little remarkable!
Re: Uniqueness Bias: Why it matters, how to curb it
#18Once we overcome uniqueness bias, we'll need to overcome '...but it might work for us' bias. Lindsay: Well, did it work for those people? Tobias: No, it never does. I mean, these people somehow delude themselves into thinking it might, but ... But it might work for us.
Isn't it socially healthy to have that bias though? Like working at a start up is statistically unlikely to pay off in a huge way, but ideally you want most founders to think they can do that so they'll make the attempt?
That means you need to pawn off startup ideas onto people with bad risk management and invest in a bunch of them to spread out your risk.
Re: Uniqueness Bias: Why it matters, how to curb it
#19Earlier quoted context omitted.
> the Metaculus community does it again and again. Do they? I can't see where on that site there is something like "history of predictions" or "track record". Edit: here. https://www.metaculus.com/questions/track-record/ But now I can't tell if it's good or bad.
The binary calibration diagram is what I would focus on. Of all the times Metaculus has said there's an x % of something happening, it has happened nearly exactly x % of the time. That is both useful and a little remarkable!
That doesn’t necessarily imply individual predictions are particularly good. If between two outcomes I say X wins 50% and I pick the winner randomly I’m going to be correct 50% of the time. However, if I offered people bets at 50/50 odds based on those predictions I would lose a great deal of money to people making more accurate predictions.
Re: Uniqueness Bias: Why it matters, how to curb it
#20Uniqueness bias is definitely a problem (see also: This time is different!) and this paper's conclusions make a lot of intuitive sense. Less competent planners are more likely to see their situation as unique. However, the measures to establish that perceived uniqueness != uniqueness seem a little bit limited to me. I've seen projects go belly-up because senior management refused to accept that the situation on the g…
>>management refused to accept ... that the situation was different
Yes! Seems that the compliment of Uniqueness Bias is Normalcy Bias. Two cliffs to fall off of on opposite sides of the path.
Makes me wonder how a similar study on projects failing due to Normalcy Bias (failing to recognize the uniqueness of a situation) would turn out, and in particular, which bias produces more and bigger cost and/or time overruns. I.e., which is the least-bad side on which to err?