I think FogBugz used this method to estimate how long open tickets would take to get closed. Or at least I remember Joel talking about it. I wonder if there are any other issues tracking apps out there making use of this technique
How to replace estimations and guesses with a Monte Carlo simulation
21–30 of 162 posts
Re: How to replace estimations and guesses with a Monte Carlo simulation
#22While a different application than shown here, my project management class taught me one of the biggest benefits of Monte Carlo simulation - estimating uncertainty. Traditionally, net present value calculations are done with single point estimates. For example, analyzing a rental property we want to buy, we'd estimate the vacancy rate, interest rate, property appreciation, maintenance expenses, and do all of that on…
A related field: https://en.wikipedia.org/wiki/Real_options_valuation
> This simple example shows how the net present value may lead the firm to take unnecessary risk, which could be prevented by real options valuation.
Even with a monte carlo on NPV, you may still make a more risky decision than needed; as decisions are rarely just "do or dont" but can be sequenced pending more information.
Re: How to replace estimations and guesses with a Monte Carlo simulation
#23I love small simulations like this, but I find tiresome writing all the scaffolding to summarize the results. So I wrote this Python module:
https://github.com/boppreh/carlo
pip install carlo
carlo "d(20)+randint(2, 5)+int(random()*20)"
This is harder than it looks, because there's too many samples to store them all, and we don't know the range of values at start. But I'm quite happy with the results.
Re: How to replace estimations and guesses with a Monte Carlo simulation
#24It would be good to build a company corpus using completed .mpp files.
Re: How to replace estimations and guesses with a Monte Carlo simulation
#25While a different application than shown here, my project management class taught me one of the biggest benefits of Monte Carlo simulation - estimating uncertainty. Traditionally, net present value calculations are done with single point estimates. For example, analyzing a rental property we want to buy, we'd estimate the vacancy rate, interest rate, property appreciation, maintenance expenses, and do all of that on…
It's not just business people. I have yet to find a task tracking system (Jira etc.) that lets you assign a range of points to a task.
People try and use nonsense like Fibonacci numbers to imply uncertainty, but then just add up all the numbers to get a number with no uncertainty measure.
Re: How to replace estimations and guesses with a Monte Carlo simulation
#26Earlier quoted context omitted.
What happens when you complete your work before you know what you need to do next? If this never happens, then you have some invisible queue, as you do have things to do next. As far as your example, that's a great example of a task that seemed like it would take long, and ended up being very very short. Can you describe why this would be bad to add into your task system? - Add Task: Run banking API in $BigBank test…
Switching to prod mode takes 5 to 7 business days, because we have to order certs from DigiCert and then upload them to $BigBank, whose team requires 5 to 7 business days to activate said certs. We expected to turn on prod once testing was finished. But we ended up discovering that prod was the only correct test environment, because their test environment is rand() and fork()ed to the point that it doesn't even sligh…
[deleted]
Re: How to replace estimations and guesses with a Monte Carlo simulation
#27It's fascinating how poor this area currently is. We are able to capture so many tiny, horrifying details about engagement and optimise shit out of ad clicking but we don't do basic things like using calendar or task dependency for internal delivery estimation.
Looking at what we're already doing on outside, improvements inside seem trivial. Good estimator should know that estimation spanning holiday season will be impacted. It should know serial dependency will span wider in time. It should know that set of unevenly distributed tasks (A-many + B-not-many) for small-team-A and larger-team-B will flow slower in time than distribution matching weighted max flow for team A and B. It should know multiple dependency on team X will have lower flow because they're maxed out.
It seems to me that monte carlo projection on timeline from inputs we already have mapped onto graph is really not a rocket science and it should have been solved problem by now. But it's not. Weird.
Re: How to replace estimations and guesses with a Monte Carlo simulation
#28I’m a big fan of the Donald Reinertsen approach: measure queue length. Simply track the time to complete each task in the team queue on average, then multiply that by the number of tasks remaining in the queue. Each team will habitually slice things into sizes they feel are appropriate. Rather than investing time to try and fail at accurately estimating each one, simply update your average every time a task is comple…
New problem: can you estimate the number of queueable tasks in this supertask?
Re: How to replace estimations and guesses with a Monte Carlo simulation
#29I really dislike posts in this vein, because they use statements like "This post will teach you how to replace estimations and guesses with a Monte Carlo simulation." as though the output of a Monte Carlo simulation were not an estimate, but some higher truth. The process of simulating many different outcomes with a touch of randomness gives the layperson the impression that we are really _doing_ something, when in f…
I sort of agree with you although for different reasons. The problem with the MC approach IMO is not really that it is also an estimate, estimates are fine. However doing a Monte Carlo simulation is only reasonable if we have a good model. So now we have moved from we can't estimate (intuitively I guess) to how to find a good model, that is not trivial and the time spend on finding and verifying the model might take…
A Monte Carlo simulation is infinitely better than all the conventional "to estimate, take your first estimate and multiply it with 10x". And even better, your manager can do that without your help and this learn better how much you tend to over/under estimate projects without trying to force you to change your estimates to fit what he think it should be.
It also ensures that the final estimate becomes on average correct, which is much better than most projects which tends to under estimate the effort it will take. Although, under estimating projects is often a feature managers wants since it makes it easier to sell it to customers, ie "My engineers estimated we can remake Twitter in a weekend, I think we should focus on that!".
Anyway, the point is that if your manager complains about your estimations being on average off then he is just incompetent and should have applied a Monte Carlo simulation on it instead of asking you to perform an easily automatable task.
Re: How to replace estimations and guesses with a Monte Carlo simulation
#30After many decades we still discuss and wonder how work amount estimates should be done and why they are always wrong. The most brilliant software engineers and project managers have developed countless different methods and nothing works. For me it tells that nature of work is such that we just can not estimate work amounts. We are just pretending that it can be done.
Absolutely; we still treat knowledge-based work as if it were factory based with known inputs and outputs (someone can chime in on MBAs etc...)