A more “back of the envelope“ approach to handling uncertainty for this sort of thing is the “three point estimate” approach, where you give a best, worst, and most likely estimate for each sub task. These points are implicitly used to parametrise a distribution, and then you analytically find the overall uncertainty, rather than through simulation.
How to replace estimations and guesses with a Monte Carlo simulation
91–100 of 162 posts
Re: How to replace estimations and guesses with a Monte Carlo simulation
#92Earlier quoted context omitted.
> A very nice thing about Monte Carlo simulation is that at the end your distribution of results are all within the feasible range. I guess Monte Carlo helps provide conservative estimates behind a façade of rigour, but the truth of the matter is that in the end it's still GIGO. Any empirical distribution only reflects the empirical measures that were used to generate it. If you bundle everything from the time it too…
I’m smiling a bit while reading your comment. You are completely correct that that estimate is virtually meaningless. And yet… that doesn’t mean it’s completely useless! I’ve been using a somewhat meaningless Monto Carlo inspired approach to project planning for a while in my consulting company. The vast majority of the projects I do have only a small amount in common with previous projects, so when I’m estimating I’…
Re: How to replace estimations and guesses with a Monte Carlo simulation
#93You don't need a monte carlo simulation if you have a known distribution. You can just add the means and variances and solve the equation rather than approximating it with a simulation. Monte carlo is more useful when drawing from an unknown distribution.
Re: How to replace estimations and guesses with a Monte Carlo simulation
#94I've actually been working on turning this idea into a product for the past year or two. Looks like I'm not the only one to have thought of this!
Re: How to replace estimations and guesses with a Monte Carlo simulation
#95Re: How to replace estimations and guesses with a Monte Carlo simulation
#96I recommend the book "Software Estimation: Demystifying the Black Art" (2006) by Steve McConnell. Nice collection of excerpts: https://leventov.medium.com/excerpts-from-software-estimatio... There is also a short chapter on estimation tools using e.g. Monte Carlo simulations. One tip which is relevant here and that is also more or less presented in the article and probably obvious :) "TIP #65: Don't treat the output…
Re: How to replace estimations and guesses with a Monte Carlo simulation
#97While a different application than shown here, my project management class taught me one of the biggest benefits of Monte Carlo simulation - estimating uncertainty. Traditionally, net present value calculations are done with single point estimates. For example, analyzing a rental property we want to buy, we'd estimate the vacancy rate, interest rate, property appreciation, maintenance expenses, and do all of that on…
One of the concepts that many people don’t get is differentiation between risk and uncertainty. These same people will then attempt to “model risk” and end up with (ironically) an “uncertain estimate” https://en.m.wikipedia.org/wiki/Knightian_uncertainty
Re: How to replace estimations and guesses with a Monte Carlo simulation
#98A much more robust approach - though not always appropriate - is a worst-case vs middle vs best-case scenario analysis
Re: How to replace estimations and guesses with a Monte Carlo simulation
#99Earlier quoted context omitted.
Yeah, FogBugz had a very similar feature 15 years ago. It was a hybrid between the approach proposed in the article, and the "always multiply by N" technique. Basically, FogBugz always asked you for your own best estimate. It tracked how much longer the actual time to completion was compared to your best estimate, and learned a simplified statistical distribution of your personal multipliers (e.g. in 80% of cases you…
I never tried FogBugz but I heard good things about it, similar to what you've written. It's surprising it didn't catch on more if it's that good? I mean, I suppose it could still catch on, the product still exists as far as I know.
Unrelated, the new owners of Kiln and FogBugz are asshats, I moved my stuff off ages ago, and I’d encourage anyone else still on them to do the same.
Re: How to replace estimations and guesses with a Monte Carlo simulation
#100### Skip to TLDR ###
Usually it's of little consequence and the stakes are low, but it's this general lack of skill in such a large population of people who are doing it poorly that gives estimation such a bad reputation in the first place. The idea of estimation is not incorrect, it's that people aren't taught how to do it properly in the first place and industry continues to propagate the myth that it's simpler than it is. Often because it's mandated by those who don't understand it themselves.
Having played in many types of industries I have a thesis that much of what we do in planning and estimating has had real benefit in some areas historically - where there are clear, repeatable, measurable observations, certain types of construction and fabrication - but not others due to their success being tied to the method alone without account for how the nature of what is being measured.
Monte Carlo has very real and valuable uses in critical path method scheduling, but, as with most things, only when used appropriately. Weather is something that fits well, because it is cyclic, we have a lot of data, we can, ahead of time, to some degree, have a likely view of where and when things like hurricanes/cyclones will occur in a given period.
In the Australian North-West Shelf where I used to plan O&G maintenance dive campaigns, knowing how long it takes to get to an area, a diver to get to depth, perform a task, return safely and leave the area is the easy part. "Everything going to plan"* we should be able to do these x number of inspections/interventions in y number of days. But weather is where you can come unstuck. Sea state matters, there is always some chance that you will have to interrupt or postpone on a day because of conditions. We know there are roughly x number of these days per cyclone season and roughly how long we might not be able to work for but pay full rates for crew, equipment etc.. An at risk period for weather goes on to the end of the schedule as contingency and is drawn down as needed, but in some scenarios where in the network of activities these multi-day events occur can have varying impacts. Inserting a three day stand-down early on could have a greater impact on the end date than a three day stand-down right near the end if at the earlier time you are doing things that must be done in a continuous sequence with - if interrupted - will need you to repeat again from the beginning. Luckily there are tools we can use to not just look at the net effects of activities in a schedule network having their own distribution curves, but also injecting events, again with their own probabilities, randomly throughout the network during the run.
This gives you a much greater insight than a 'P90' estimate on your critical path, it gives you the sensitivity of activities in your network.
### TLDR ###
This all assumes, of course, that what you have modelled (planned) reflects reality to some degree. A bad schedule with bad bounds will give you a bad result regardless. It doesn't matter how many times you simulate it.
*it rarely does