How to replace estimations and guesses with a Monte Carlo simulation

How to replace estimations and guesses with a Monte Carlo simulation
lucasfcosta.com

----

Sample a few activities your team has completed. Check how long the 90 % smallest activities are on average, and compare it to the average of the biggest 10 %. Or the median compared to the maximum, or whatever. You'll probably find the difference is about an order of magnitude or less. In the grand scheme of things, every activity is the same size. You can estimate it as exp(mean(log(size)) and be within an order of magnitude almost every time.

Once your team has accepted that something is "an" activity and not a set of activities, don't bother estimating. For all practical intents, size is effectively constant at that point. What matters is flow, not size.

----

For the above sample, also study how long passed between the "go" decision on the task and when it was actually released to customers. In a stable team, this number will be eerily close to the theoretical based on Little's law referenced in the parent comment.

Oh, and you shouldn't focus on man-hours. Work with calendar days. Not only does that simplify mental arithmetic for everyone, it's also the only thing that matters in the end. Your customer couldn't care less that you finished their functionality in "only" 6 man-hours if it took you six weeks to get it through your internal processes.

----

Fun follow-up to the size experiment: now ask someone intimately familiar with your customers to estimate the dollar value of each activity. You might find that while all activities are practically the same size, they'll have very different dollar values. That's what you ought to be estimating.

Traditionally, net present value calculations are done with single point estimates. For example, analyzing a rental property we want to buy, we'd estimate the vacancy rate, interest rate, property appreciation, maintenance expenses, and do all of that on a cash flow time-adjusted basis.

There's variability on every aspect of the through, and understanding how much can easily swing your decision.

Traditional methodology is to pull the trigger on the action if the net present value is greater than 0. We put in a variety of estimated factors, ran it through Crystal Ball (simulation software), and found a 75% chance the NPV would be negative. It suddenly isn't the safe investment it looked like originally.

---

In practice, it's tough to get people in business to understand uncertainty. They gravitate towards "the number", which is almost surely wrong all the time.

For example, at my day job my task is to implement banking. The day to day tasks change... day to day. There aren't a "number of tasks remaining in the queue," since whatever I'm doing is what I'm doing.

One could say this is poor planning. But due to the nature of Big Banks, each task is usually blocking the next one -- in other words, it's not possible to discover or plan what you need to do next, until you've finished currently.

An example of this is when we realized we didn't need to run our banking API using $BigBank's test environment. Their test environment was ... uh ... well, let's just say, when we realized that we could simply switch on "production mode" and bypass their test environment altogether, we collectively facepalmed while rejoicing.

It wouldn't be possible to add "switch to the production environment" into the queue several days ago, because we didn't discover that we could do that until yesterday during our biweekly sync call.

I'm sympathetic to your writeup, and I like your recommended approach. But I just wanted to point out a realistic case of it failing. But in fairness, I think every estimation approach would fail us, so don't feel singled out. :)

Perhaps your approach will work in most cases though, and I'm merely stuck in a twilight zone special case.