Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Uniqueness Bias: Why it matters, how to curb it

arxiv.org

Post reply on HN
Print view View original

21–30 of 70 posts

Re: Uniqueness Bias: Why it matters, how to curb it

#21

Post by SiempreViernes »

nemomarx wrote: post #10
sdenton4 wrote: post #6

Once we overcome uniqueness bias, we'll need to overcome '...but it might work for us' bias. Lindsay: Well, did it work for those people? Tobias: No, it never does. I mean, these people somehow delude themselves into thinking it might, but ... But it might work for us.

Isn't it socially healthy to have that bias though? Like working at a start up is statistically unlikely to pay off in a huge way, but ideally you want most founders to think they can do that so they'll make the attempt?

This sounds quite a lot like "trick people into taking risks I don't want to take myself", not sure that's bound to produce socially good outcomes...

Re: Uniqueness Bias: Why it matters, how to curb it

#22

Post by pif »

ozgune wrote: post #3

I read this book called "How Big Things Get Done." I've seen my fair share of projects going haywire and I wanted to understand if we could do better. The book identifies uniqueness bias as an important reason for why most big projects overrun. (*) In short, this bias leads planners to view their projects as unique, thereby disregarding valuable lessons from previous similar projects. (*) The book compiles 16,000 big…

> 99.5% of those projects overrun their timeline or budget.

This does not shock me. In the corporate world, plans are not there to be adhered to, but only to give upper management a feeling of having tightened the rope for those pesky engineers who wanted to work at a lazy pace. Such feeling usually vanishes as soon as reality kicks in.

Re: Uniqueness Bias: Why it matters, how to curb it

#23

Post by joebob42 »

tomlue wrote: post #8
kqr wrote: post #5

Earlier quoted context omitted.

This goes beyond project estimations. The Metaculus tournaments have questions like > Will YouTube be banned in Russia before October 1? These are the kinds of questions most people I meet would claim are impossible to forecast accurately on because of their supposed uniqueness, yet the Metaculus community does it again and again. I believe the problem is a lack of statistical literacy. Back in the 1500s shipping ins…

I know you're coming from a good place, but the 'most people I meet don't understand this' line about statistics is quite arrogant. Most people you meet are fully capable of understanding statistics; you should do a better job of explaining it when it comes up, or maybe you are the one who misunderstands. After all, most statisticians thought Marilyn vos Savant was wrong about the goats too...

My experience is that most people don't understand statistics and can be pretty easily mislead. That includes myself, with my only defense being that I'm at least aware statistics don't intuitively make sense and either ignore arguments based on them or if absolutely necessary invest the extra time to properly understand.

Most people either don't realize this is necessary or don't have the background to do it even if they did, in my experience.

Re: Uniqueness Bias: Why it matters, how to curb it

#24

Post by kqr »

krisoft wrote: post #15
kqr wrote: post #5

Earlier quoted context omitted.

This goes beyond project estimations. The Metaculus tournaments have questions like > Will YouTube be banned in Russia before October 1? These are the kinds of questions most people I meet would claim are impossible to forecast accurately on because of their supposed uniqueness, yet the Metaculus community does it again and again. I believe the problem is a lack of statistical literacy. Back in the 1500s shipping ins…

> Back in the 1500s shipping insurance was priced under the assumption that if you just learned enough about the voyage you could tell for certain whether it would be successful or not. Do you have a source on that? That sounds very unlikely to me. People just have to look at a single storm to see that it sometimes destroys some ships and does not destroy others. It is very clearly has a luck component. How could hav…

Okay, that was an oversimplification. I have long wanted to write more on this but it never comes out right. What they did was determine the outcome with certainty barring the will of God or similar weasels.

I.e. they did thorough investigations and then determined whether the shipment ought to be successful or not, and then the rest laid in the hands of deities or nature, in ways they assumed were incalculable.

This normative perspective of what the future ought to be like is typical of statistical illiteracy.

I'll get back to you on sources after dinner!

Re: Uniqueness Bias: Why it matters, how to curb it

#25

Post by JohnCClarke »

Vecr wrote: post #18
nemomarx wrote: post #10

Earlier quoted context omitted.

Isn't it socially healthy to have that bias though? Like working at a start up is statistically unlikely to pay off in a huge way, but ideally you want most founders to think they can do that so they'll make the attempt?

That means you need to pawn off startup ideas onto people with bad risk management and invest in a bunch of them to spread out your risk.

If only there was some kind of social media devoted to gathering all those folks in one place. Oh, wait

Re: Uniqueness Bias: Why it matters, how to curb it

#26

Post by kqr »

Retric wrote: post #19
kqr wrote: post #17

Earlier quoted context omitted.

The binary calibration diagram is what I would focus on. Of all the times Metaculus has said there's an x % of something happening, it has happened nearly exactly x % of the time. That is both useful and a little remarkable!

That doesn’t necessarily imply individual predictions are particularly good. If between two outcomes I say X wins 50% and I pick the winner randomly I’m going to be correct 50% of the time. However, if I offered people bets at 50/50 odds based on those predictions I would lose a great deal of money to people making more accurate predictions.

This is true! The Metaculus community forecasts also performs very well (in terms of score) in tournaments including other aggregation mechanisms though.

Re: Uniqueness Bias: Why it matters, how to curb it

#27

Post by lordnacho »

tomlue wrote: post #8
kqr wrote: post #5

Earlier quoted context omitted.

This goes beyond project estimations. The Metaculus tournaments have questions like > Will YouTube be banned in Russia before October 1? These are the kinds of questions most people I meet would claim are impossible to forecast accurately on because of their supposed uniqueness, yet the Metaculus community does it again and again. I believe the problem is a lack of statistical literacy. Back in the 1500s shipping ins…

I know you're coming from a good place, but the 'most people I meet don't understand this' line about statistics is quite arrogant. Most people you meet are fully capable of understanding statistics; you should do a better job of explaining it when it comes up, or maybe you are the one who misunderstands. After all, most statisticians thought Marilyn vos Savant was wrong about the goats too...

You don't have to be on the internet for long to see:

- "Polls are useless, they only sampled a few thousand people"

- "Why do we need the crime figures adjusted for the age/income/etc groups? Just gimme the raw truth!"

Have to say, I think stats are the least well taught area in the math curriculum. Most people by far have no clue what Simpson's or Berkson's paradoxes are. Most people do not have the critical sense when presented with stats to ask questions like "how was the data collected" or "does it show what we think it shows".

I just don't see it, tough ironically I don't have stats to back it up.

Re: Uniqueness Bias: Why it matters, how to curb it

#28

Post by marcus0x62 »

krisoft wrote: post #15
kqr wrote: post #5

Earlier quoted context omitted.

This goes beyond project estimations. The Metaculus tournaments have questions like > Will YouTube be banned in Russia before October 1? These are the kinds of questions most people I meet would claim are impossible to forecast accurately on because of their supposed uniqueness, yet the Metaculus community does it again and again. I believe the problem is a lack of statistical literacy. Back in the 1500s shipping ins…

> Back in the 1500s shipping insurance was priced under the assumption that if you just learned enough about the voyage you could tell for certain whether it would be successful or not. Do you have a source on that? That sounds very unlikely to me. People just have to look at a single storm to see that it sometimes destroys some ships and does not destroy others. It is very clearly has a luck component. How could hav…

Somewhat related, but very detailed, is the book Against the Gods[0] by Peter Bernstein that documents the historical development of understanding probability and risk. It also discusses what people believed before these concepts were understood.

0 - https://www.amazon.com/Against-Gods-Remarkable-Story-Risk/dp...

Re: Uniqueness Bias: Why it matters, how to curb it

#29

Post by organsnyder »

lordnacho wrote: post #27
tomlue wrote: post #8

Earlier quoted context omitted.

I know you're coming from a good place, but the 'most people I meet don't understand this' line about statistics is quite arrogant. Most people you meet are fully capable of understanding statistics; you should do a better job of explaining it when it comes up, or maybe you are the one who misunderstands. After all, most statisticians thought Marilyn vos Savant was wrong about the goats too...

You don't have to be on the internet for long to see: - "Polls are useless, they only sampled a few thousand people" - "Why do we need the crime figures adjusted for the age/income/etc groups? Just gimme the raw truth!" Have to say, I think stats are the least well taught area in the math curriculum. Most people by far have no clue what Simpson's or Berkson's paradoxes are. Most people do not have the critical sense…

You don't have to be on the Internet long to see flat-earthers or any number of asinine ways of thinking. You can't stretch discrete observations from a supremely massive sample size into "most people".

Re: Uniqueness Bias: Why it matters, how to curb it

#30

Post by steveBK123 »

ozgune wrote: post #3

I read this book called "How Big Things Get Done." I've seen my fair share of projects going haywire and I wanted to understand if we could do better. The book identifies uniqueness bias as an important reason for why most big projects overrun. (*) In short, this bias leads planners to view their projects as unique, thereby disregarding valuable lessons from previous similar projects. (*) The book compiles 16,000 big…

I think a lot of modern US public works fall prey to politicians who think the objective of the project is the spending of the money. That is - they push for spending on transit so they can talk about how much, in dollars, they got passed in transit funding. The actual outcomes for many of them are, at best, inconsequential.

Further cynicism could be layered in if you consider some of the blocks of donors (infrastructure contractors / RE devs / etc) and blocks of voters (union transit workers & construction workers) who are recipients again of the spending but not the outcomes.

Finally a lot of the problems come from a long gap of not doing capital projects and so hollowing out of state capacity which has been outsourced. If you outsource your planning, they are less incentivized to re-use existing cookie-cutter plans for subway stations. If you outsource your project management, they are less incentivized to keep costs down. ETc.

Post reply on HN