Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

If You Say Something Is “Likely,” How Likely Do People Think It Is?

hbr.org

Post reply on HN
Print view View original

61–70 of 116 posts

Re: If You Say Something Is “Likely,” How Likely Do People Think It Is?

#61

Post by archgoon »

safgasCVS wrote: post #55

"Lesson 1: Use probabilities instead of words to avoid misinterpretation." Probabilities are meaningless unless it’s a repeatable experiment otherwise its a ludic fallacy eg "There's a 70% chance of Hillary winning". This is an un-provable statement. Either she wins and prediction was right, or she loses and it counts as part of the 30%. This is Nate Silver's get-out-of-jail-free card so even when he's wrong he comes…

> Probabilities are meaningless unless it’s a repeatable experiment otherwise its a ludic fallacy

Um... this goes against the entirety of the Bayesian approach to statistics. I think you'd find a lot of very intelligent people who disagree strongly with this statement.

The Bayesian approach takes probabilities as subjective confidences. You can describe confidences as "well calibrated" if, when you look at their historical guesses, if their 70% assessments are correct 70% of the time.

Re: If You Say Something Is “Likely,” How Likely Do People Think It Is?

#62

Post by a13n »

sanbor wrote: post #49

In my experience, putting numbers to your gut feeling probabilities causes people to not take them seriously. Example: "I think there is 30% chances that I will get a promotion next month". If I say 30% people ask how I came up with that number. If I say "not very likely" then it gets accepted as an educated guess. Probably because we're used that those numbers must come from some dataset. Or should I say "I think th…

30% is specific, as if it's the result of a calculation in a spreadsheet. But with something like getting a promotion, how would you calculate that so specifically? You can't. That's why people don't take it seriously.

"Not very likely" is a rough estimate, like "probably not", or "less than 30% likely". It's a lot more believable that with the information you know, you're able to predict with that certainty.

Re: If You Say Something Is “Likely,” How Likely Do People Think It Is?

#63

Post by Infernal »

arendtio wrote: post #54
crispyambulance wrote: post #11

It depends on the context. Hillary Clinton had a >70% of winning the US presidential election according to the most responsible analyses (see 538: https://projects.fivethirtyeight.com/2016-election-forecast/ ). Most folks took 70% to mean that she would certainly win and were bitterly disappointed the morning after. On the other hand no sane person would (willingly) play Russian roulette with a 70% or even 5 of 6 cha…

> In everyday life, one conflates probability with severity of outcome. My everyday example for that is the weather forecast and the question 'will it rain' often answered with a precipitation probability? - Probability: How likely is it that I will be hit by at least one rain drop - Severity: How many rain drops will hit me It sounds a little abstract, but whenever I see some everyday weather forecast I wonder what…

From http://www.wesh.com/article/what-does-a-chance-of-rain-reall...

The "Probability of Precipitation" (PoP) describes the chance of precipitation occurring at any point you select in the area. How do forecasters arrive at this value? Mathematically, PoP is defined as follows: PoP = C x A where "C" = the confidence that precipitation will occur somewhere in the forecast area, and where "A" = the percent of the area that will receive measurable precipitation, if it occurs at all. So... in the case of the forecast above, if the forecaster knows precipitation is sure to occur (confidence is 100%), he/she is expressing how much of the area will receive measurable rain. (PoP = "C" x "A" or "1" times ".4" which equals .4 or 40%) But, most of the time, the forecaster is expressing a combination of degree of confidence and areal coverage. If the forecaster is only 50% sure that precipitation will occur, and expects that, if it does occur, it will produce measurable rain over about 80 percent of the area, the PoP (chance of rain) is 40%. ( PoP = .5 x .8 which equals .4 or 40%. )

Re: If You Say Something Is “Likely,” How Likely Do People Think It Is?

#64

Post by 3pt14159 »

edbaskerville wrote: post #40
OscarCunningham wrote: post #34

Earlier quoted context omitted.

From a Bayesian perspective, or from a betting one, it doesn't make sense to put probabilities in a confidence interval. You might be uncertain about the world, but you can be certain about how much uncertainty you have, since it's a property of your own mind.

This is worth emphasizing. If you have a model with a parameter that represents the probability of something happening, and then you have uncertainty in that parameter, your real probability needs to integrate over that uncertainty. But the final probability is still just a single number. But I suspect the GP's reasoning was getting interference from the valid point that poll numbers , which are not probabilities, ar…

Hmm. Doesn't it depend on whether re-rolls are allowed as to how confident we can be?

For example, I can imagine a well engineered coin that has a 0.5 probability of heads with a 0.005 confidence (ie, we suspect that the true weighting between heads and tails is likely between 0.495 and 0.505 19 times out of 20) and I can contrast it with a hastily made coin that can still have a 0.5 expected probability with a 0.1 confidence (ie, we expect the true probability to be between 0.4 to 0.6, 19 times out of 20).

Coin A and Coin B have dramatically different impacts on our decision making. For example, selling insurance against 5 identical flips in a row is a much more expensive proposition for Coin B than Coin A.

I feel like I really should know this given my data science background, but sometimes the basics slip away.

Re: If You Say Something Is “Likely,” How Likely Do People Think It Is?

#65

Post by OscarCunningham »

TTPrograms wrote: post #58
OscarCunningham wrote: post #34

Earlier quoted context omitted.

From a Bayesian perspective, or from a betting one, it doesn't make sense to put probabilities in a confidence interval. You might be uncertain about the world, but you can be certain about how much uncertainty you have, since it's a property of your own mind.

It might be of some general interest to note that there exist such objects - distributions over distributions, or metaprobabilities. They have practical use - for example, if you are playing a game with uncertainty, you might observe something in the next step that changes your belief, or distribution over world state, b = P(s). If you have some expected distribution of next observations then you can talk about the p…

Something similar certainly is useful in the case of flipping a coin. Suppose we have two coins, one of which we know is fair and one of which we know is biased but we don't know which way. Then our knowledge about the bias of the first coin would be described by a distribution with a sharp peak around 0.5, but the distribution for the second coin would have two lumps either side of 0.5. Then the probability of heads on the next flip would be 0.5 for either coin. But if someone asked us to bet on the probability of heads on the next two flips, then the second coin would have a higher probability than the first. The different distributions for the two coins don't make a difference for one flip, but do for multiple flips.

But there's no meaningful way to repeat an election, so I don't think similar distributions or confidence intervals are useful in that case.

EDIT: We can imagine rerolling an election, but since we can't actually do it we don't have to bet on it so the information wouldn't be very useful to us.

Re: If You Say Something Is “Likely,” How Likely Do People Think It Is?

#66

Post by williamscales »

computerphage wrote: post #59
DINKDINK wrote: post #31

Earlier quoted context omitted.

>Hillary Clinton had a >70% of winning the US presidential election Probabilities without confidence intervals[1] are by-and-large meaningless (She has a 90% chance of of winning with a confidence interval of +11% -100%). No amount of d3.js on 538's blog will change this. https://en.wikipedia.org/wiki/Confidence_interval

> Probabilities without confidence intervals are by-and-large meaningless. That's just not true. If I believe that my team has a 20% chance to win and you offer me a bet with anything better than 5-to-1 odds I should take the bet. If you offer me anything worse than 5-to-1, then I should not take the bet. There's no fuzz factor necessary; no confidence interval that I need to use to make the decision. Perhaps you're…

I think this is getting to the root of the problem — you're taking a perfectly valid frequentist view of probability, that is, viewing it as a series of discrete experiments. But the probabilities that were assigned to Clinton's victory were derived from Bayesian probability theory that estimates the likelihood that event occurs based on empirically determined prior probabilities that contribute to that event. Those prior distributions have uncertainty which leads to uncertainty in the resulting prediction.

I apologize if I've explained this poorly, this is just my layman's understanding of the matter.

Re: If You Say Something Is “Likely,” How Likely Do People Think It Is?

#67

Post by jcrites »

This article, and some of the comments here, reminded me of another article (and GDC talk) by Civilization game designer Sid Meier, about his experience with players' perception of probability in games.

Sid's talk grapples with the issue "If the game says you have 3-to-1 odds to win a battle, how often do players actually expect to win?"

> When designing the combat system in Civilization: Revolution, Sid Meier found himself up against some interesting design problems. His players didn't understand math. In Civ Rev, the strength of units were displayed up front to players before battle to show the odds of victory. For example, an attacking unit might be rated at 1.5 with the defending unit at 0.5. This is a 3-to-1 situation.

> Unfortunately, the testers expected to win this battle every time despite there being a 25% chance of losing each time. Sid tweaked the math to make the player win more in this situation. Next, the reverse case was tested. The player had 1-to-3 odds. If they won, the math was functioning properly. They had a slim chance to win and they did.

> Sid identified a few cases of interest. When the player was presented with 3-to-1 or 4-to-1 odds, they expect to win. With 2-to-1 odds, the player will accept losing some of the time, but expect to win at 20-to-10, which is just a larger expression of 2-to-1. When the numbers get larger, the perceived advantage grows.

http://www.shacknews.com/article/62807/sid-meier-and-rob-par...

Here is a link to the actual GDC talk by Sid with the content about probability: https://youtu.be/bY7aRJE-oOY?t=18m22s

(It's true. When playing a Civ game, it's not fun to have a 10-to-1 strength advantage and lose the battle anyway.)

In later games, Sid removed probability from the game in favor of an outcome that's predictable. Instead, each unit will be damaged in proportion to the ratio of the units' strengths (or something like that).

Re: If You Say Something Is “Likely,” How Likely Do People Think It Is?

#68

Post by OscarCunningham »

3pt14159 wrote: post #64
edbaskerville wrote: post #40

Earlier quoted context omitted.

This is worth emphasizing. If you have a model with a parameter that represents the probability of something happening, and then you have uncertainty in that parameter, your real probability needs to integrate over that uncertainty. But the final probability is still just a single number. But I suspect the GP's reasoning was getting interference from the valid point that poll numbers , which are not probabilities, ar…

Hmm. Doesn't it depend on whether re-rolls are allowed as to how confident we can be? For example, I can imagine a well engineered coin that has a 0.5 probability of heads with a 0.005 confidence (ie, we suspect that the true weighting between heads and tails is likely between 0.495 and 0.505 19 times out of 20) and I can contrast it with a hastily made coin that can still have a 0.5 expected probability with a 0.1 c…

Right. Both coins have a probability of 0.5 on the next flip, but when you do multiple flips the hastily made coin has a higher chance of a result with lots of heads or lots of tails.

We could do the same thing for the 2016 election, but we would have to specify exactly what we meant by a "repeat". Do we just let Hillary run for the 2020 election and see what happens? Or do we put back every atom to the exact position it had in 2014, so that the only divergences between the two elections are caused by quantum randomness? Or something in between, like looking at all elections where a demagogue outsider runs against an established insider?

Really it doesn't matter what definition of "repeat" we choose. Since the repeat won't actually happen, we can't be called to bet on it, so knowing the probability in that case isn't too useful. Whereas a coin actually can be flipped multiple times.

Re: If You Say Something Is “Likely,” How Likely Do People Think It Is?

#70

Post by OscarCunningham »

evandijk70 wrote: post #53
crispyambulance wrote: post #11

It depends on the context. Hillary Clinton had a >70% of winning the US presidential election according to the most responsible analyses (see 538: https://projects.fivethirtyeight.com/2016-election-forecast/ ). Most folks took 70% to mean that she would certainly win and were bitterly disappointed the morning after. On the other hand no sane person would (willingly) play Russian roulette with a 70% or even 5 of 6 cha…

Most people, bookmakers and statisticians thought Hillary Clinton had more than 90% chance to win the electronisch - Nate Silver was aan outlier

I think the betting odds were also around 70:30 on the day before the election. The newspapers were almost all way off though.
Post reply on HN