Live data from Hacker News

How to replace estimations and guesses with a Monte Carlo simulation

lucasfcosta.com

81–90 of 162 posts

Re: How to replace estimations and guesses with a Monte Carlo simulation

#81

I really dislike posts in this vein, because they use statements like "This post will teach you how to replace estimations and guesses with a Monte Carlo simulation." as though the output of a Monte Carlo simulation were not an estimate, but some higher truth. The process of simulating many different outcomes with a touch of randomness gives the layperson the impression that we are really _doing_ something, when in f…

> It erects a straw person

Just wanted to appreciatively note your use of inclusive, non-gendered language here.

Re: How to replace estimations and guesses with a Monte Carlo simulation

#82

Code used Rust, interesting, I have come to expect Python and just recently used the open source version of Matlab called Octave.

Author here. I thought no one was going to make comments about Rust, but I just loved using it for this post. Thanks, Peter :)

I have been looking for an excuse to use Rust, thank you for the interesting post on all counts...

Re: How to replace estimations and guesses with a Monte Carlo simulation

#83
post #53

Worth checking out Guesstimate[0] for a neat tool to help make these kinds of estimations. [0] - https://www.getguesstimate.com/

Great tool. I've used Crystal Ball in the past, and this is so much easier to use.

Re: How to replace estimations and guesses with a Monte Carlo simulation

#84

For those who like to do things a little more visually I recommend the open-sourced Argo[0] for microsoft excel. There's a lot you can do with a monte carlo model. Tornado plots are something I highly recommend for understanding risk profiles around any complex task [0] https://github.com/boozallen/argo/releases

This looks great. I wish there were something like this for google sheets or that worked outside windows. Guesstimate is close to this.

Re: How to replace estimations and guesses with a Monte Carlo simulation

#85
> let die = Uniform::from(1..7);

I think this is a good example of the bad consequences of zero-based indexing. First, I hope you agree that a line of code describing a six-sided die should never feature a literal 7!

But zero-based indexing causes us to embrace exclusive ranges: if we want 6 numbers, we write 0..6 and that’s fine.

If you want 6 numbers starting at 1, then you must use an inclusive range operator, since you don’t want to bring the irrelevant number 7 into the picture. Now Rust has that: ..= and the author should use it here. But e.g. Python does not. Personally in Python, I would still refuse to write 7 and write range(1, 6+1) instead, but that’s most people would write range (1, 7).

Which brings me to my point: I’m not trying to argue against zero-based indexing, but I believe that

(a) programming language designers (e.g. Python) don’t recognize sufficiently that it has downsides as well as upsides, and languages need to provide things like inclusive ranges, even if they don’t actually provide 1-based indexing, since 1-based indexing scenarios do naturally occur when programming. It seems that Julia is an example of a language that has very much recognized that both sides have merit.

(b) In the standard case of a language that basically adopts zero-based indexing (most modern languages), 1-based thinking should not be presented as somehow vulgar or ignorant. A good test of whether your language is too hardline on zero-based indexing is: would you ever see users of your language using the literal 7 when describing a six-sided die? If you see that happening, the language is being too hardline on zero-based indexing; either you don't provide inclusive ranges or they're not promoted enough in the documentation.

Re: How to replace estimations and guesses with a Monte Carlo simulation

#86
post #12

Earlier quoted context omitted.

The real danger is not understanding that this "uncertainty" estimate is a function of your assumptions. How you model the distribution of your inputs is huge, and often not stated clearly. GIGO

> The real danger is not understanding that this > "uncertainty" estimate is a function of your assumptions. The bigger danger, in my experience, is treating estimates as deadlines . Unfortunately, that seems to be the norm in almost every place that I'm familiar with.

“Don’t worry, this is _purely_ an estimate and would never be used to set the deadline. So just give us something to share with the ELT…”

Re: How to replace estimations and guesses with a Monte Carlo simulation

#88
You don't need a monte carlo simulation if you have a known distribution. You can just add the means and variances and solve the equation rather than approximating it with a simulation. Monte carlo is more useful when drawing from an unknown distribution.

Re: How to replace estimations and guesses with a Monte Carlo simulation

#89

Earlier quoted context omitted.

It's not just business people. I have yet to find a task tracking system (Jira etc.) that lets you assign a range of points to a task. People try and use nonsense like Fibonacci numbers to imply uncertainty, but then just add up all the numbers to get a number with no uncertainty measure.

Story points are a curse on the software development industry. However much people say "they're indicative, and don't map to hours", someone, somewhere, will map them to hours. The most accurate project plan I was ever involved in had only 3 values that could be assigned to a piece of work during the early estimation phase: hours, days, and weeks. Each of those was then turned into a range of possible hours they coul…

That seems reasonable, but I would say you don't have to resort to such crudeness. In my experience I know the difference between a task that will definitely take weeks and a task that might take a day or might take weeks. Just let me write that down!

Even if you don't have any idea about the uncertainty we already have a crude way of measuring it - planning poker! Just record everyone's guesses instead of throwing away the uncertainty information. There's a huge difference between everyone guessing 5, and some people guessing 1 and others guessing 20.

I agree about points being stupid though - there's simply no way to avoid it being converted to/from time, because that's the actual unit of work.

Re: How to replace estimations and guesses with a Monte Carlo simulation

#90
Many commenters on this story believe the main purpose of estimation is to assert when something will be done. Then they say "this is impossible" and call the whole exercise a waste of time, or management abuse.

I gotta tell you that unless you're working with absolute bozos, nobody is looking at estimates and saying "oh duh, I am betting the farm this will complete on that date."

Benefits of estimation that are more important that the date itself:

Enabling trade-off analysis: If you tell me the thing will be done in two months, I bucket the thing in my head as something that takes "a few months" as opposed to "weeks" or "years." Often that is enough to drive a build/pass decision on the feature.

Dependency visualization: If I don't ask you to estimate, you might just start coding. Estimation may force you to think about critical paths and dependencies, then we can plan for them to improve your chances of success.

Troubleshooting: why did we miss an estimate? A task was harder than we thought, that's ok. We missed a dependency, ok that's a prompt to think about those better next time. Developer keeps getting distracted, maybe we need to change something on the team. Estimation is just "calling a shot" which then enables us to (maybe) learn from missing it.

All those things are valuable even if the actual estimated date is missed, even by a lot.

Post reply on HN