Live data from Hacker News

Why software projects take longer than you think – a statistical model

erikbern.com

41–50 of 324 posts

Re: Why software projects take longer than you think – a statistical model

#41
post #15

I once had a really convoluted metaphor for estimation which involved opening boxes that sometimes contained other boxes which sometimes contained other boxes... I wonder how that models mathematically.

Well, in the most extreme form, you'd wind-up with something like the poisson distribution [1], where your chance of finishing each week would be the same.

This is not entirely implausible but it produces effects even more paradoxical/pathological than log-normal. Here, you could give a correct estimate of the expected time for a project to complete as 50 weeks. You could then reach week fifty and again give a correct estimate of 50 more weeks required. And then you could finish the next week.

[1] https://en.wikipedia.org/wiki/Poisson_distribution

Re: Why software projects take longer than you think – a statistical model

#42
The interesting thing is that by the central limit theorem, the mean of a mean is normally distributed. This is extremely helpful. Here's what I suggest you do:

Same size your stories to small values. Do 30 stories in a sprint and take the mean. Do 30 sprints and take the mean of the sprint. What you get is the mean amount of time to do a sprint of 30 stories. What's amazing is that this estimate will be normally distributed. You can measure the variance to get error bars.

Of course 900 stories to get good estimates ;-) However, imagine that your stories averaged 2 days each. Imagine as well that you have a team of 10 people. That means that you will finish a "sprint" of 30 stories in 6 days (on average). 30 sprints is 180 days -- the better part of a year, but you probably don't need a 95% confidence interval.

You will find that after a few sprints, you'll be able to predict the sprint length pretty well (or if you set your sprints to be a certain size, then you will predict the number of stories that will fit in it, with error bars).

The other cool thing is that by doing this, you will be able to see when stories are outliers. This is a highly undervalued ability IMHO. Once a story passes the mean plus the variance, you know you've got a problem. Probably time to replan. If you have a group of stories that are exceeding that time, then you may have a systemic estimation problem (often occurs when personnel change or some kind of pressure is being applied to the team). This kind of early warning system allows you to start trying to find potential problems.

This is really the secret behind "velocity" or "load factor" in XP. Now, does it work on a normal team? In my experience, it doesn't because groups of people are crap at calmly using statistics to help them. I've had teams where they were awesome at doing it, but that was the minority, unfortunately.

Re: Why software projects take longer than you think – a statistical model

#43
post #7

Earlier quoted context omitted.

An approach that gives very similar results to yours but is more scientific is to multiply the initial estimates by e (2.718). Or, if you're conservative, then by pi.

The above approach uses the factor 2.2. Why would your factors be any better?

Because e is more scientific.

Re: Why software projects take longer than you think – a statistical model

#44
post #18

US navy has developed something similar with beta statistical distribution. You estimate "Optimistic", "Most likely" and "Pessimistic" time estimates for each task in the project and then use beta distribution on it. Some tasks take way longer than estimated. Here is the link to the time estimation described above with Beta distribution. https://www.isixsigma.com/methodology/project-management/bet...

I do this but only with best and worst case. Any estimate with a worst case of more than 5 days need to be broken down again unless it's a known quantity. Thanks for the link, I had no idea this was part of 6 sigma

Re: Why software projects take longer than you think – a statistical model

#45
post #28
post #4

I believe this is the reason why scrum uses story points instead of time estimate. By putting uncertainty on the same level as effort, you give it more weight. And using a fibonacci sequence rather than a continuous amount with the rule you should round up if unsure tend to correct those defects.

there is an uncharitable response to this thread, which is marked dead. I do think there is a kernel of truth in that response, which is nothing more than mocking. why fibonacci? I think it's reasonable to say time includes an exponential error. an estimated 1 hour task is very different than an estimated 1 week task. i see that 1 hour, 1 day and 1 week estimates are progressively, and likely exponentially, worse. Is…

The reason the Fibonacci sequence is used is because we don't think in an incremental scale but in a logarithm scale. In other words, we think that some simple takes 1 day, a more complex takes 3 and a even more complex takes 5 days. Usually we don't jump from 1 to 2 days when thinking about a task effort.

Re: Why software projects take longer than you think – a statistical model

#46
post #4

I believe this is the reason why scrum uses story points instead of time estimate. By putting uncertainty on the same level as effort, you give it more weight. And using a fibonacci sequence rather than a continuous amount with the rule you should round up if unsure tend to correct those defects.

If regular time estimation has tricky statistical properties when combining multiple estimates, story point estimation is just hopeless. Especially on a Fibonacci scale. When estimating, nobody thinks that an 8-point story is the same as eight 1-point stories. Even the metric itself is nonlinear!

Re: Why software projects take longer than you think – a statistical model

#47

An important aspect of being a professional software engineer is having the backbone to sometimes say things like: - “I don’t know yet enough about the problem to give you even a rough estimate. If you’d like, I can take a day to dig into it and then report back.” - “This first part should take 2-3 days. 5 on the outside. But the second part relies heavily on an API whose documentation and error messages are in Chine…

The other thing you can always refer back to is that the accuracy of an estimate increases the closer it is to finishing it - that is, it's very inaccurate at the start. This is one of the pillars of the agile way of working - you and your team can give a fairly accurate estimation for what can be finished within a week or two. Not so much for a year.

The longer estimations are very much due to changing priorities and a developing environment and understanding of what is being built though, hence the inaccuracy.

Re: Why software projects take longer than you think – a statistical model

#48
post #35

An important aspect of being a professional software engineer is having the backbone to sometimes say things like: - “I don’t know yet enough about the problem to give you even a rough estimate. If you’d like, I can take a day to dig into it and then report back.” - “This first part should take 2-3 days. 5 on the outside. But the second part relies heavily on an API whose documentation and error messages are in Chine…

100% this. It's not your job to say something that will make your client happy. It's your job to tell them your honest assessment of the task.

Imagine a tradie saying it'd take 1week to finish remodelling your kitchen, and then it turns out they couldn't. As a professional, it's your responsibility to educate the client on the issue - most technical problem could be explained to a layman if the context is set up properly.

The problem occurs when the client doesn't want to hear it.

Re: Why software projects take longer than you think – a statistical model

#49
In the world of small medium projects often the major issue is that software engineers give estimates for writing the software but customers take that to mean time to actual delivery in production and a lot of the time have no idea how big a task deployment and integration are...or don't even have a plan for that.

Re: Why software projects take longer than you think – a statistical model

#50

An important aspect of being a professional software engineer is having the backbone to sometimes say things like: - “I don’t know yet enough about the problem to give you even a rough estimate. If you’d like, I can take a day to dig into it and then report back.” - “This first part should take 2-3 days. 5 on the outside. But the second part relies heavily on an API whose documentation and error messages are in Chine…

>But when you advocate clearly for your needs as a professional, people are generally reasonable.

You should do all this but there will still be times when people are unreasonable, it's just human nature.

Post reply on HN