How to replace estimations and guesses with a Monte Carlo simulation

In order to replace estimation, we are trying Basecamp's Shapeup language and techniques [1]

In a nutshell the thinking is reversed, instead of asking:

- "how long would it take to implement X?"

you ask:

- "how much appetite (in weeks and people involved) do I have for this feature ?"

Then you work with a fix "dead line" and variable scope.

Instead of user stories, they call it Bets, and they give themselves 2 weeks to shape bets and 6 to implement them (these two cycles can happen in parallel), during those weeks is about leaving the design at the Mockup/sketch level, and identify with help of business and technical people all the possible risks, so the Bet safe becomes a safer Bet.

We are still in the first cycle, and still a lot of questions and doing our own little customisations, but everyone response seems to be positive with the way of working, curious if anyone else has tried succeed and fail.

- [1] https://basecamp.com/shapeup

The author first talks about the waste of time that is upfront analysis/planning or adding a buffer for contingency and how Monte Carlo simulation fixes for that, but then the caveats seem to be based on an established team, of a fixed size doing similar style work.

For me, this stage of a project has never faced big hiccups in estimation & our teams are normally quite good at knowing "what's a 2 vs a 5 point story" — if there _is_ a big discussion in a planning meeting at that stage, it's normally because one person has overlooked a key requirement (or the other is assuming a requirement that doesn't exist) so it's a good opportunity to solve for that.

To use the analogy, we're probably quite good in established teams at estimating how long it'll take to write 10 blog posts. The trouble comes when someone says "The latest blog posts aren't generating enough interest. Can you estimate how long it'd take for you to write 10 poems instead? We want a mix of iambic pentameter and maybe some trochaic tetrameter (TBC). We've gotten budget approval to increase the size of the team, so we'll have 5 new people (off-shore) joining next week so we can go faster. Also, some of the old blog posts seem to be getting lots of visitors in South America, so we may need to translate these to Portuguese/Spanish too. When can you give me an estimate for all that?"

There is also a short chapter on estimation tools using e.g. Monte Carlo simulations. One tip which is relevant here and that is also more or less presented in the article and probably obvious :) "TIP #65: Don't treat the output of a software estimation tool as divine revelation. Sanity-check estimation tool outputs just as you would other estimates."

I guess Monte Carlo helps provide conservative estimates behind a façade of rigour, but the truth of the matter is that in the end it's still GIGO.

Any empirical distribution only reflects the empirical measures that were used to generate it. If you bundle everything from the time it took employee A to walk the dog while allocated to project Foo to the time it took employee Z to fix a nasty Eisenbug while allocated to project Bar, and Foo and Bar used totally different tech stacks and team members and even approaches to project planning, that distribution is meaningless in estimating, say, how much time it will take employee G to implement a React widget.