Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Uniqueness Bias: Why it matters, how to curb it

arxiv.org

Post reply on HN
Print view View original

51–60 of 70 posts

Re: Uniqueness Bias: Why it matters, how to curb it

#51

Post by isaacfrond »

kqr wrote: post #36
kqr wrote: post #24

Earlier quoted context omitted.

Okay, that was an oversimplification. I have long wanted to write more on this but it never comes out right. What they did was determine the outcome with certainty barring the will of God or similar weasels. I.e. they did thorough investigations and then determined whether the shipment ought to be successful or not, and then the rest laid in the hands of deities or nature, in ways they assumed were incalculable. This…

I think I got most of this from Willful Ignorance (Weisberg, 2014) but it was a while since I read it (and this was before I made more detailed notes) so I might be misremembering. There are what looks like fantastic books on the early days of probability and statistics aside from Weisberg (Ian Hacking is an author that comes to mind, and maybe Stephen Stigler?) but I have not yet taken the time to read them – I'll r…

Could it be related to Huygens publication in 1657? (De Ratiociniis in Ludo Aleae)

Re: Uniqueness Bias: Why it matters, how to curb it

#52

Post by inopinatus »

roenxi wrote: post #4

Uniqueness bias is definitely a problem (see also: This time is different!) and this paper's conclusions make a lot of intuitive sense. Less competent planners are more likely to see their situation as unique. However, the measures to establish that perceived uniqueness != uniqueness seem a little bit limited to me. I've seen projects go belly-up because senior management refused to accept that the situation on the g…

That’s why it’s a bias, not a fallacy.

Re: Uniqueness Bias: Why it matters, how to curb it

#53

Post by carlmr »

ozgune wrote: post #3

I read this book called "How Big Things Get Done." I've seen my fair share of projects going haywire and I wanted to understand if we could do better. The book identifies uniqueness bias as an important reason for why most big projects overrun. (*) In short, this bias leads planners to view their projects as unique, thereby disregarding valuable lessons from previous similar projects. (*) The book compiles 16,000 big…

>In short, this bias leads planners to view their projects as unique, thereby disregarding valuable lessons from previous similar projects.

I once had a discussion. We had some managers that would systematically over or underestimate their projects, mostly underestimate. I suggested that we take into account the estimation accuracy of previous projects for that manager and adjust their estimate.

They said that each project is too unique to do this. But I saw the same optimism or pessimism playing out repeatedly for the same managers when looking at the numbers.

Although tbf I think accurate estimation can be bad if the person managing the project knows about the estimation. Since if they have more time, there's less pressure, and they'll have overruns again, making the estimation inaccurate again.

Hofstadter's law, to me, is less about accurate estimation and more about human psychology. If you know you have more time, you waste more time.

This is also a failure-mode in agile project management. If you don't have strict deadlines, it's easy to fall into infinite iteration, because there's always something that could be done better.

Re: Uniqueness Bias: Why it matters, how to curb it

#54

Post by michaelmarkell »

tomlue wrote: post #8
kqr wrote: post #5

Earlier quoted context omitted.

This goes beyond project estimations. The Metaculus tournaments have questions like > Will YouTube be banned in Russia before October 1? These are the kinds of questions most people I meet would claim are impossible to forecast accurately on because of their supposed uniqueness, yet the Metaculus community does it again and again. I believe the problem is a lack of statistical literacy. Back in the 1500s shipping ins…

I know you're coming from a good place, but the 'most people I meet don't understand this' line about statistics is quite arrogant. Most people you meet are fully capable of understanding statistics; you should do a better job of explaining it when it comes up, or maybe you are the one who misunderstands. After all, most statisticians thought Marilyn vos Savant was wrong about the goats too...

The Monty hall problem is a great example of something I’ve been educated into believing, rationalizing, whatever you want to call it…but I would still never claim I “understand it.” I think that’s maybe the source of disagreement here, there are many truly unintuitive outcomes of statistics that are not “understood” by most people in the most respectful sense of the word, even if we’ve been educated into knowing the formula, knowing how to come to the right answer, etc.

It’s like in chess, I know that the Sicilian is a good opening, that I’m supposed to play a6 in the najdorf, but I absolutely do not “understand” the Najdorf, and I do think it’s fundamentally past the limit of most humans understanding.

Re: Uniqueness Bias: Why it matters, how to curb it

#55

Post by LegionMammal978 »

christianqchung wrote: post #44
LegionMammal978 wrote: post #43

Earlier quoted context omitted.

It also seems gameable: for every big question of societal importance that people care about for its own sake, have a thousand random little questions where the outcome is dead obvious and can be predicted trivially. Would you know anything talking about weighing questions to account for this?

Trivial questions wouldn't result in a good histogram where a probability of 30% actually results in something happening roughly 1 in 3 times. Trivial would mean questions where the community forecast is 1% or 99%. Those are not the vast majority of questions on the site. It would be very boring if the site was 70% questions where the answer is obviously yes or obviously no. Additionally, many questions require that…

I'd also include under 'trivial' things like "Will this 6-sided die roll a 1?", or really any other well-understood i.i.d. process whose distribution of outcomes never changes under reasonable circumstances. Not just things that are 0.1% or 99.9%.

> Who would be gaming the system here anyway, the site? Individual users?

Cynically speaking, users would be incentivized to ask and answer more trivial questions to pad out their predictive accuracy, so they can advertise that accuracy elsewhere for social clout.

Re: Uniqueness Bias: Why it matters, how to curb it

#56

Post by jakobloekke »

ozgune wrote: post #3

I read this book called "How Big Things Get Done." I've seen my fair share of projects going haywire and I wanted to understand if we could do better. The book identifies uniqueness bias as an important reason for why most big projects overrun. (*) In short, this bias leads planners to view their projects as unique, thereby disregarding valuable lessons from previous similar projects. (*) The book compiles 16,000 big…

Cannot recommend that book enough

Re: Uniqueness Bias: Why it matters, how to curb it

#57

Post by krisoft »

kqr wrote: post #36
kqr wrote: post #24

Earlier quoted context omitted.

Okay, that was an oversimplification. I have long wanted to write more on this but it never comes out right. What they did was determine the outcome with certainty barring the will of God or similar weasels. I.e. they did thorough investigations and then determined whether the shipment ought to be successful or not, and then the rest laid in the hands of deities or nature, in ways they assumed were incalculable. This…

I think I got most of this from Willful Ignorance (Weisberg, 2014) but it was a while since I read it (and this was before I made more detailed notes) so I might be misremembering. There are what looks like fantastic books on the early days of probability and statistics aside from Weisberg (Ian Hacking is an author that comes to mind, and maybe Stephen Stigler?) but I have not yet taken the time to read them – I'll r…

Thank you for the reference!

Re: Uniqueness Bias: Why it matters, how to curb it

#58

Post by [deleted] »

ozgune wrote: post #3

I read this book called "How Big Things Get Done." I've seen my fair share of projects going haywire and I wanted to understand if we could do better. The book identifies uniqueness bias as an important reason for why most big projects overrun. (*) In short, this bias leads planners to view their projects as unique, thereby disregarding valuable lessons from previous similar projects. (*) The book compiles 16,000 big…

[deleted]

Re: Uniqueness Bias: Why it matters, how to curb it

#59

Post by sebstefan »

roenxi wrote: post #4

Uniqueness bias is definitely a problem (see also: This time is different!) and this paper's conclusions make a lot of intuitive sense. Less competent planners are more likely to see their situation as unique. However, the measures to establish that perceived uniqueness != uniqueness seem a little bit limited to me. I've seen projects go belly-up because senior management refused to accept that the situation on the g…

"We are facing unique challenges with our very sophisticated CRUD app. It is distributed and must maintain high reliability."

Re: Uniqueness Bias: Why it matters, how to curb it

#60

Post by shever73 »

kqr wrote: post #39
tomlue wrote: post #8

Earlier quoted context omitted.

I know you're coming from a good place, but the 'most people I meet don't understand this' line about statistics is quite arrogant. Most people you meet are fully capable of understanding statistics; you should do a better job of explaining it when it comes up, or maybe you are the one who misunderstands. After all, most statisticians thought Marilyn vos Savant was wrong about the goats too...

It's definitely arrogant. But after much experience trying to explain these things to people, I'm more and more convinced it's not just "if you just put it the right way they will understand". Sure, they will nod politely and pretend to understand, and may even do a passable job of reciting it, but once its cast in a slightly different light they are just as confused. Much as with reading and writing, I think it take…

What resources would you recommend for someone who wants to improve their statistical literacy? You mention reading the right books, I'd appreciate it if you could give a short list, if you have time.
Post reply on HN