Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Unsure Calculator – back-of-a-napkin probabilistic calculator

filiph.github.io

Post reply on HN
Print view View original

71–80 of 170 posts

Re: Show HN: Unsure Calculator – back-of-a-napkin probabilistic calculator

#71

Post by lswainemoore »

pyfon wrote: post #64
SamBam wrote: post #61

Earlier quoted context omitted.

I don't quite understand, probably because my math isn't good enough. If you're treating -1~1 as a normal distribution, then it's centered on 0. If you're working out the answer using a Monte Carlo simulation, then you're going to be testing out different values from that distribution, right? And aren't you going to be more likely to test values closer to 0? So surely the most likely outputs should be far from 0, rig…

Only 1 percent of values would end up being 100+ on a uniform distribution. For normal it is higher but maybe not much more so.

That may be true, but if you look at the distribution it puts out for this, it definitely smells funny. It looks like a very steep normal distribution, centered at 0 (ish). Seems like it should have two peaks? But maybe those are just getting compressed into one because of resolution of buckets?

Re: Show HN: Unsure Calculator – back-of-a-napkin probabilistic calculator

#72

Post by montag »

lorenzowood wrote: post #70

See also Guesstimate https://getguesstimate.com . Strengths include treating label and data as a unit, a space for examining the reasoning for a result, and the ability to replace an estimated distribution with sample data => you can build a model and then refine it over time. I'm amazed Excel and Google Sheets still haven't incorporated these things, years later.

Thank you, I would have mentioned this myself, but forgot the name of it.

Re: Show HN: Unsure Calculator – back-of-a-napkin probabilistic calculator

#73

Post by etbebl »

explosion-s wrote: post #66

I made one that's much faster because it instead modifies the normal distribution instead of sending thousands of samples: https://gistpreview.github.io/?757869a716cfa1560d6ea0286ee1b...

This is more limited. I just tested and for one example, exponentiation seems not to be supported.

Re: Show HN: Unsure Calculator – back-of-a-napkin probabilistic calculator

#75

Post by etbebl »

pyfon wrote: post #64
SamBam wrote: post #61

Earlier quoted context omitted.

I don't quite understand, probably because my math isn't good enough. If you're treating -1~1 as a normal distribution, then it's centered on 0. If you're working out the answer using a Monte Carlo simulation, then you're going to be testing out different values from that distribution, right? And aren't you going to be more likely to test values closer to 0? So surely the most likely outputs should be far from 0, rig…

Only 1 percent of values would end up being 100+ on a uniform distribution. For normal it is higher but maybe not much more so.

OK, but do we necessarily just care about the central 95% range of the output? This calculation has the weird property that values in the tails of the input correspond to values in the middle of the output, and vice versa. If you follow the intuition that the range you specify in the input corresponds to the values you expect to see, the corresponding outputs would really include -inf and inf.

Now I'm realizing that this doesn't actually work, and even in more typical calculations the input values that produce the central 95% of the output are not necessarily drawn from the 95% CIs of the inputs. Which is fine and makes sense, but this example makes it very obvious how arbitrary it is to just drop the lowermost and uppermost 2.5%s rather than choosing any other 95/5 partition of the probability mass.

Re: Show HN: Unsure Calculator – back-of-a-napkin probabilistic calculator

#76

Post by jrowen »

roughly wrote: post #44

I like this! In the grand HN tradition of being triggered by a word in the post and going off on a not-quite-but-basically-totally-tangential rant: There’s (at least) three areas here that are footguns with these kinds of calculations: 1) 95% is usually a lot wider than people think - people take 95% as “I’m pretty sure it’s this,” whereas it’s really closer to “it’d be really surprising if it were not this” - by and…

This jives with my general reaction to the post, which was that the added complexity and difficulty of reasoning about the ranges actually made me feel less confident in the result of their example calculation. I liked the $50 result, you can tack on a plus or minus range but generally feel like you're about breakeven. On the other hand, "95% sure the real balance will fall into the -$60 to +$220 range" feels like it's creating a false sense of having more concrete information when you've really just added compounding uncertainties at every step (if we don't know that each one is definitely 95%, or the true min/max, we're just adding more guesses to be potentially wrong about). That's why I don't like the Drake equation, every step is just compounding wild-ass guesses, is it really producing a useful number?

Re: Show HN: Unsure Calculator – back-of-a-napkin probabilistic calculator

#77

Post by kqr »

I have made a similar tool but for the command line[1] with similar but slightly more ambitious motivation[2].

I really like that more people are thinking in these terms. Reasoning about sources of variation is a capability not all people are trained in or develop, but it is increasingly important.[3]

[1]: https://git.sr.ht/~kqr/precel

[2]: https://entropicthoughts.com/precel-like-excel-for-uncertain...

[3]: https://entropicthoughts.com/statistical-literacy

Re: Show HN: Unsure Calculator – back-of-a-napkin probabilistic calculator

#78

Post by kqr »

jrowen wrote: post #76
roughly wrote: post #44

I like this! In the grand HN tradition of being triggered by a word in the post and going off on a not-quite-but-basically-totally-tangential rant: There’s (at least) three areas here that are footguns with these kinds of calculations: 1) 95% is usually a lot wider than people think - people take 95% as “I’m pretty sure it’s this,” whereas it’s really closer to “it’d be really surprising if it were not this” - by and…

This jives with my general reaction to the post, which was that the added complexity and difficulty of reasoning about the ranges actually made me feel less confident in the result of their example calculation. I liked the $50 result, you can tack on a plus or minus range but generally feel like you're about breakeven. On the other hand, "95% sure the real balance will fall into the -$60 to +$220 range" feels like it…

It is producing a useful number. As more truly independent terms are added, error grows with the square root while the point estimation grows linearly. In the aggregate, the error makes up less of the point estimation.

This is the reason Fermi estimation works. You can test people on it, and almost universally they get more accurate with this method.

If you got less certain of the result in the example, that's probably a good thing. People are default overconfident with their estimated error bars.

Re: Show HN: Unsure Calculator – back-of-a-napkin probabilistic calculator

#79

Post by kqr »

Nevermark wrote: post #65
btilly wrote: post #52

Earlier quoted context omitted.

I strongly agree with this, and particularly point 1. If you ask people to provide estimated ranges for answers that they are 90% confident in, people on average produce roughly 30% confidence intervals instead. Over 90% of people don't even get to 70% confidence intervals. You can test yourself at https://blog.codinghorror.com/how-good-an-estimator-are-you/ .

From link: > Heaviest blue whale ever recorded I don't think estimation errors regarding things outside of someone's area of familiarity say much. You could ask a much "easier"" question from the same topic area and still get terrible answers: "What percentage of blue whales are blue?" Or just "Are blue whales blue?" Estimating something often encountered but uncounted seems like a better test. Like how many cars pas…

I have no familiarity with blue whales but I would guess they're 1--5 times the mass of lorries, which I guess weigh like 10--20 cars which I in turn estimate at 1.2--2 tonnes, so primitively 12--200 tonnes for a normal blue whale. This also aligns with it being at least twice as large as an elephant, something I estimate at 5 tonnes.

The question asks for the heaviest, which I think cannot be more than three times the normal weight, and probably no less than 1.3. That lands me at 15--600 tonnes using primitive arithmetic. The calculator in OP suggests 40--320.

The real value is apparently 170, but that doesn't really matter. The process of arriving at an interval that is as wide as necessary but no wider is the point.

Estimation is a skill that can be trained. It is a generic skill that does not rely on domain knowledge beyond some common sense.

Post reply on HN