Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How to replace estimations and guesses with a Monte Carlo simulation

lucasfcosta.com

Post reply on HN
Print view View original

61–70 of 162 posts

Re: How to replace estimations and guesses with a Monte Carlo simulation

#61

Post by strife25 »

I used this method for a while on one of my teams. It was useful and helped provide more accurate estimates.

Some tips I have for people interested:

* Get about two months of velocity data in place first. * Run the simulation on a regular basis. Update your estimates every week/sprint. * It helps to have individual work units be "sized" as consistent as you can. (E.g. each ticket is 2-3 days, a consistent pointing system, etc.) It's ok if anomalies happen from time to time. * Dates will get more accurate the further you get into a project.

Some things I've learned from doing this exercise:

* When work units are sized to be 2-3 days of effort, engineers complete about 4 units/wk on average. * Don't shoot for the perfect estimate at the start of a project. Plan as much scope as you know and enough to get started. * As new scope is found, update your project plan and estimates.

Re: How to replace estimations and guesses with a Monte Carlo simulation

#62

Post by zmk_ »

edu-ap wrote: post #15
asdf_snar wrote: post #4

I really dislike posts in this vein, because they use statements like "This post will teach you how to replace estimations and guesses with a Monte Carlo simulation." as though the output of a Monte Carlo simulation were not an estimate, but some higher truth. The process of simulating many different outcomes with a touch of randomness gives the layperson the impression that we are really _doing_ something, when in f…

A model is not required if you use data from your team's past performance with a tool like this: https://marketplace.atlassian.com/apps/1216661/actionableagi...

Which is called bootstrapping.

Re: How to replace estimations and guesses with a Monte Carlo simulation

#63

Post by maigret »

kqr wrote: post #3
brightball wrote: post #2

I’m a big fan of the Donald Reinertsen approach: measure queue length. Simply track the time to complete each task in the team queue on average, then multiply that by the number of tasks remaining in the queue. Each team will habitually slice things into sizes they feel are appropriate. Rather than investing time to try and fail at accurately estimating each one, simply update your average every time a task is comple…

Strongly seconding this. For anyone still hesitant, I further recommend the following experiments: ---- Sample a few activities your team has completed. Check how long the 90 % smallest activities are on average, and compare it to the average of the biggest 10 %. Or the median compared to the maximum, or whatever. You'll probably find the difference is about an order of magnitude or less. In the grand scheme of thing…

This. I've found that, whatever the project is, the velocity of a team in terms what they define as a "task" is pretty constant, surprisingly so even. In the end, just counting the outstanding tasks proved to be a good estimator of where we'd end up at the deadline.

Re: How to replace estimations and guesses with a Monte Carlo simulation

#64

Post by ChrisMarshallNY »

Loic wrote: post #18
Ntrails wrote: post #12

Earlier quoted context omitted.

The real danger is not understanding that this "uncertainty" estimate is a function of your assumptions. How you model the distribution of your inputs is huge, and often not stated clearly. GIGO

GIGO, the first thing I learnt, as I entered the industry 20 years ago. This was from a 60 year old engineer who told me that experience is only a nice name for "all the @#$% I made I will try not to make again". A very nice thing about Monte Carlo simulation is that at the end your distribution of results are all within the feasible range. If you do error propagation using uncertainty on your parameters, you can get…

> experience is only a nice name for "all the @#$% I made I will try not to make again"

One of my favorite quotes is one that was made famous by Will Rogers and Rita Mae Brown:

"Good judgment comes from experience. Experience comes from bad judgment."

I have seen this "attributed to Nasrudin," which means no one really knows who came up with it, the first time.

Re: How to replace estimations and guesses with a Monte Carlo simulation

#65

Post by dognotdog »

A nice exposition! I think Omniplan does this [1], and it certainly helped me in the past get a better idea on variability of long-term project planning. Of course, it's only as good as the assumptions that go into it!

[1] https://www.omnigroup.com/omniplan/features/#Monte-Carlo-Sim...

Re: How to replace estimations and guesses with a Monte Carlo simulation

#66

Post by markmaglana »

strife25 wrote: post #61

I used this method for a while on one of my teams. It was useful and helped provide more accurate estimates. Some tips I have for people interested: * Get about two months of velocity data in place first. * Run the simulation on a regular basis. Update your estimates every week/sprint. * It helps to have individual work units be "sized" as consistent as you can. (E.g. each ticket is 2-3 days, a consistent pointing sy…

If you already have ~2 months worth of data, why not just use that to plot a distribution curve? Seems to me that the monte carlo simulation that the article describes is really just used to "guess" the likely distribution curve. Seems unnecessary if you already have ~2 months worth of data.

Re: How to replace estimations and guesses with a Monte Carlo simulation

#67

Post by qubex »

For anybody who wants to try it out in a spreadsheet environment and/or is looking for a useful abandoned project to duplicate, I recommend Risk Engine for Mac. Unfortunately it hasn’t been maintained for a long time and has been abandoned, but I keep using it.

https://www.macupdate.com/app/mac/30431/risk-engine

Re: How to replace estimations and guesses with a Monte Carlo simulation

#69

Post by qznc »

brightball wrote: post #2

I’m a big fan of the Donald Reinertsen approach: measure queue length. Simply track the time to complete each task in the team queue on average, then multiply that by the number of tasks remaining in the queue. Each team will habitually slice things into sizes they feel are appropriate. Rather than investing time to try and fail at accurately estimating each one, simply update your average every time a task is comple…

That is essentially what a Burndown Chart does in visual way, isn’t it?

https://en.m.wikipedia.org/wiki/Burn_down_chart

Post reply on HN