How to replace estimations and guesses with a Monte Carlo simulation • Hacker News Board

Re: How to replace estimations and guesses with a Monte Carlo simulation

#141

Post by listenallyall » Wed, Oct 06, 2021, 8:05 PM UTC

dagw wrote: post #114

xyzelement wrote: post #90

Many commenters on this story believe the main purpose of estimation is to assert when something will be done. Then they say "this is impossible" and call the whole exercise a waste of time, or management abuse. I gotta tell you that unless you're working with absolute bozos, nobody is looking at estimates and saying "oh duh, I am betting the farm this will complete on that date." Benefits of estimation that are more…

I gotta tell you that unless you're working with absolute bozos, nobody is looking at estimates and saying "oh duh, I am betting the farm this will complete on that date." Happens all the time, and not just by bozos. Sometimes getting paid is entirely dependent on delivering when you said you would. I used to work at an animation studio and one year we worked on a series of Christmas themed TV spots. When we said we…

That's not an estimate, it's a deadline. Different concepts.

Re: How to replace estimations and guesses with a Monte Carlo simulation

#142

Post by nanis » Wed, Oct 06, 2021, 9:26 PM UTC

mateo411 wrote: post #121

nanis wrote: post #103

Earlier quoted context omitted.

> Monte Carlo simulation - estimating uncertainty. By definition, "uncertainty" is the thing that does not have a PDF ... You can quantify risk based on assumptions you make about underlying probability distributions from which you are drawing. Far too often, even in Monte Carlo, people decide to work with the easy distributions instead of the most appropriate distributions. The "easy" distributions tend to admit act…

I would think that you could have "uncertainty" even if you do have a PDF. Maybe there is a formal definition of uncertainty that I am not aware of. The PDF can describe the uncertainty for every outcome.

Risk is what you can put a probability on (i.e., quantify). Uncertainty is what you can't. Uncertainty encompasses unknown unknowns. See Knight[1] and Keynes[2].

Good rule of thumb even if it sounds overly simplified.

[1]: https://www.econlib.org/library/Knight/knRUP.html

[2]: https://www.jstor.org/stable/4538116

Re: How to replace estimations and guesses with a Monte Carlo simulation

#143

Post by 8note » Wed, Oct 06, 2021, 10:19 PM UTC

xyzelement wrote: post #90

Many commenters on this story believe the main purpose of estimation is to assert when something will be done. Then they say "this is impossible" and call the whole exercise a waste of time, or management abuse. I gotta tell you that unless you're working with absolute bozos, nobody is looking at estimates and saying "oh duh, I am betting the farm this will complete on that date." Benefits of estimation that are more…

Explicitly asking for a critical path sounds better than hoping somebody will consider it while doing something else.

Re: How to replace estimations and guesses with a Monte Carlo simulation

#144

Post by toss1 » Wed, Oct 06, 2021, 10:19 PM UTC

xyzelement wrote: post #126

toss1 wrote: post #111

Earlier quoted context omitted.

>>I gotta tell you that unless you're working with absolute bozos, nobody is looking at estimates and saying "oh duh, I am betting the farm this will complete on that date." I hate to break it to you, but it happens a lot (and yes, management structures can often work to filter absolute bozos upwards). They make all kinds of representations to customers, investors, etc. about dates that have nothing approaching this…

Eh, I think there is two ways this can go. If you're really working with people who are 'so stupid' as to take estimates literally and make plans assuming guaranteed success in those times, then you probably have other leadership/management/risk problems in your company and are doomed anyway. There are plenty of time where there's a little bit of managerial art to communicating estimates to customers and prospects. F…

You are making the assumption that development estimates have anything to do with when management, sales, or customers decide that they need something.

Even without mgt/sales over-promising stuff to customers based on wishful thinking and only wanting to say "Yes" to a stupid customer demand without even bothering to ask development/engineering, I've seen situations where the date is the date, period.

I've been in plenty of situations where it doesn't matter - it's an event, scheduled years in advance, national television, and the show goes on at Monday 16:00 PDT, period. Scheduling estimates have nothing to do with it. Be ready and be there, or everyone gets nothing. The only question is: "Can the team do it?".

Re: How to replace estimations and guesses with a Monte Carlo simulation

#145

Post by charles_f » Wed, Oct 06, 2021, 10:48 PM UTC

sillysaurusx wrote: post #7

brightball wrote: post #2

I’m a big fan of the Donald Reinertsen approach: measure queue length. Simply track the time to complete each task in the team queue on average, then multiply that by the number of tasks remaining in the queue. Each team will habitually slice things into sizes they feel are appropriate. Rather than investing time to try and fail at accurately estimating each one, simply update your average every time a task is comple…

What do you do when your future tasks are unknown or ambiguous? For example, at my day job my task is to implement banking. The day to day tasks change... day to day. There aren't a "number of tasks remaining in the queue," since whatever I'm doing is what I'm doing. One could say this is poor planning. But due to the nature of Big Banks, each task is usually blocking the next one -- in other words, it's not possible…

> The day to day tasks change... day to day. There aren't a "number of tasks remaining in the queue," since whatever I'm doing is what I'm doing.

What you are describing is not ambiguity, it's total variability. If your future is 100% random, it is, by definition, impossible to predict. Such a state would also mean a total absence of direction/vision. Predicting dates is not only impossible but not a question you can ask, since you don't know what's next.

What I'm going to challenge is that you're effectively in such a case, because I don't think it's true.

> One could say this is poor planning. [...] in other words, it's not possible to discover or plan what you need to do next, until you've finished currently. [...] because we didn't discover that we could do that until yesterday during our biweekly sync call.

The example you're giving *is* poor planning. You're going into execution without validating base assumptions. That you discover the specifics of a dependency that late into the game is that you're going into it without a plan. I'm not judging, in your case maybe no-one is asking for any sort of accountability, and just executing is the best recourse with the lower overhead. But the fact that you can't estimate isn't due to the environment, it's due to the fact that you don't have a plan. Some of the companies I worked with are fine with that, most are not.

Re: How to replace estimations and guesses with a Monte Carlo simulation

#146

Post by myth2018 » Wed, Oct 06, 2021, 10:50 PM UTC

game_the0ry wrote: post #80

myth2018 wrote: post #74

I think it's understandable why programmers dread doing estimates so much. But I believe there are multiple factors contributing to the current overall mess and the guilt should be shared among several areas -- and that includes us. Estimates are often, if not always, mistaken as commitments or deadlines. But we don't do a great job on explaining that estimates are probabilistic by nature, there are risks involved, a…

> Estimates are often, if not always, mistaken as commitments or deadlines This. This is the problem with "estimates." My solution - and this will sound bad - is to over-estimate.

I don't judge, and I recur to this solution as well, more often than I'd like to. But it doesn't only sounds bad.. it is bad indeed, for a number of reasons.

For instance, work usually expands to spend all the time available. Then, if you estimate a one-month task as four months, you will likely not win three months to spend on other tasks. You'll probably spend the whole four months on a one-month task instead.

Another issue: if you are competing for a project, over-estimates will translate in higher than necessary costs, and you may end up overpricing a project and missing the opportunity.

However, as I said, I don't judge and I also recur to that more often than not. On many Organizations, there aren't many other ways to deal with the matter. And that's why I say that the entire Organization looses: managers will have a good night of sleep after putting all the burden over your shoulders, but the losses are affecting every single stakeholder.

It's about time for them to start behaving a bit more responsibly.

Re: How to replace estimations and guesses with a Monte Carlo simulation

#147

Post by markmaglana » Thu, Oct 07, 2021, 1:39 AM UTC

xyzelement wrote: post #90

Many commenters on this story believe the main purpose of estimation is to assert when something will be done. Then they say "this is impossible" and call the whole exercise a waste of time, or management abuse. I gotta tell you that unless you're working with absolute bozos, nobody is looking at estimates and saying "oh duh, I am betting the farm this will complete on that date." Benefits of estimation that are more…

I don't understand why commenters are so hung up on the "unless you're working with absolute bozos" part. That sentence is irrelevant compared to the rest of the comment. The three benefits listed are a great articulation of how estimates should be used. Couldn't have said it any better.

Re: How to replace estimations and guesses with a Monte Carlo simulation

#148

Post by _y5hn » Thu, Oct 07, 2021, 3:40 AM UTC

listenallyall wrote: post #141

dagw wrote: post #114

Earlier quoted context omitted.

I gotta tell you that unless you're working with absolute bozos, nobody is looking at estimates and saying "oh duh, I am betting the farm this will complete on that date." Happens all the time, and not just by bozos. Sometimes getting paid is entirely dependent on delivering when you said you would. I used to work at an animation studio and one year we worked on a series of Christmas themed TV spots. When we said we…

That's not an estimate, it's a deadline. Different concepts.

Not really. You have to estimate within the deadline. Estimates are often required due to such dependencies.

Re: How to replace estimations and guesses with a Monte Carlo simulation

#149

Post by _y5hn » Thu, Oct 07, 2021, 3:44 AM UTC

nanis wrote: post #101

xyzelement wrote: post #90

Many commenters on this story believe the main purpose of estimation is to assert when something will be done. Then they say "this is impossible" and call the whole exercise a waste of time, or management abuse. I gotta tell you that unless you're working with absolute bozos, nobody is looking at estimates and saying "oh duh, I am betting the farm this will complete on that date." Benefits of estimation that are more…

While the article is rather low quality in and of itself, it still manages to point out: > Before claiming that the simulation above is reliable, I must expose a few things you’ll need to pay attention to when using a Monte Carlo Simulation to make forecasts: > (A) The quality of the inputs upon which your simulations depend. > (B) The consistency and predictability of your team. > (C) The need to re-forecast. > (D)…

We're stuck in the eternal loop: "Tech Y will save us from ourselves and having to think."

Re: How to replace estimations and guesses with a Monte Carlo simulation

#150

Post by _y5hn » Thu, Oct 07, 2021, 4:07 AM UTC

charles_f wrote: post #145

sillysaurusx wrote: post #7

Earlier quoted context omitted.

What do you do when your future tasks are unknown or ambiguous? For example, at my day job my task is to implement banking. The day to day tasks change... day to day. There aren't a "number of tasks remaining in the queue," since whatever I'm doing is what I'm doing. One could say this is poor planning. But due to the nature of Big Banks, each task is usually blocking the next one -- in other words, it's not possible…

> The day to day tasks change... day to day. There aren't a "number of tasks remaining in the queue," since whatever I'm doing is what I'm doing. What you are describing is not ambiguity, it's total variability. If your future is 100% random, it is, by definition, impossible to predict. Such a state would also mean a total absence of direction/vision. Predicting dates is not only impossible but not a question you can…

It's like this for day-day operations when leadership is absent and no CSI ever gets prioritized over new development. You can say the org is dysfunctional, but there's less leverage workers can use to change such situations. Especially when efficiency measures get rewarded with layoffs.