Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Poor developer estimates, a.k.a. guesstimation

alexefish.com

Post reply on HN
Print view View original

61–70 of 72 posts

Re: Poor developer estimates, a.k.a. guesstimation

#61

Post by jacques_chester »

kabdib wrote: post #45
lobotryas wrote: post #5

Earlier quoted context omitted.

You are absolutely right. It's impossible to estimate how long it will take to complete something you have never done before in your life. Unless the new code being written is absolutely trivial, estimates are going to be guesses, at best. The only half-decent solution I found so far is iterative development. Take a feature and break it down i to big rocks. Further break down the big rocks into little chunks - each t…

Decomposing as far as you can is a good practice ... you'll still be way off, because you think you know all the problems that you're going to need to solve. So about halfway through the project your list is going to be longer and management is going to be freaking out. The elephant in the room: Debugging. I don't know how many times I've been asked, "When are you going to find that bug?" Sometimes they are very easy…

> you'll still be way off, because you think you know all the problems that you're going to need to solve

Absolutely; but I think it's reasonable to assume that a more accurate estimate is a more valuable estimate. You can also track estimate error over time and use that to adjust future estimates.

> The elephant in the room: Debugging. I don't know how many times I've been asked, "When are you going to find that bug?"

The problem is that in any sample of n=1, variability is huge. It's like saying "I am going to pick one person out of the global population. How tall is he or she?"

Well of course I can't know that. The sample is too small. But if instead I am told "we are going to pick ten thousand people at random from the global population; how tall are they in total?" then I can use previously-gathered statistics to give a range of estimates that will probably be accurate.

Re: Poor developer estimates, a.k.a. guesstimation

#62

Post by jacques_chester »

phillmv wrote: post #48
ExpiredLink wrote: post #43

Earlier quoted context omitted.

> Estimations are based on experience, yet developers are constantly building brand new types of products. If all you built was "blog engines" over and over again, you would get damned good at estimating them. The trick in software is as soon as something needs to be reused over and over again, it is abstracted, put in a library, built into a framework, so you can make your product different (read: build something en…

And even then, large capital projects are routinely wildly off estimates. I think the delta in "how far off we were" just tends to be larger more frequently.

In the Industrial Megaprojects book I quoted elsewhere, the single best predictor of every measure of project performance was "Front End Loading" -- how completely the project has been studied and specified before it is funded. There are at least three FEL stages. Companies that muck up FEL-1 are doomed to failure, companies that muck up FEL-2 or FEL-3 are doomed to poor returns.

Re: Poor developer estimates, a.k.a. guesstimation

#63

Post by JoeAltmaier »

jacques_chester wrote: post #30
JoeAltmaier wrote: post #26

Earlier quoted context omitted.

I don't know, in software projects the country is not just unknown but potentially unknowable; no-one may have walked that kind of terrain before. THEN there are no experts, its useless to predict. And diving deeper into the problem may involve actually solving it. That's my issue with software planning. I can often DO the damn project in the time it takes to estimate doing it. What responsible course of action can I…

All estimates are wrong, what matters is improving their business value by making them more accurate. That some estimates will be less accurate due to uncertainty is not by itself a reason to totally abandon estimating. The generalised case of your argument is that estimation is imperfect in all cases, therefore, we must abandon estimation. This is known as the Nirvana Fallacy: "we have a partial, imperfect solution.…

Wasn't talking about uncertainty at all, so I'm not sure why that comment. And no, its not really cool to pretend I said something else so you can debunk that.

I'll repeat myself then: the task of diving into a problem to estimate its time is, for many problems, approximately equal to the time to solve the problem. And I honestly have a hard time figuring out a responsible approach in those cases.

Re: Poor developer estimates, a.k.a. guesstimation

#64

Post by jared314 »

jacques_chester wrote: post #59
ExpiredLink wrote: post #43

Earlier quoted context omitted.

> Estimations are based on experience, yet developers are constantly building brand new types of products. If all you built was "blog engines" over and over again, you would get damned good at estimating them. The trick in software is as soon as something needs to be reused over and over again, it is abstracted, put in a library, built into a framework, so you can make your product different (read: build something en…

> But how long does it take to climb a never-climbed mountain? This estimate could be derived in a number of ways. By analogy to other mountains, by creating a statistical model of climbing time based on parameters like gradient and climber skill, or by aggregating expert judgements of the mountain's likely climbing time. A better example of a difficult-to-estimate task was the Manhattan Project. How long would it ta…

> but we are almost never performing fundamental research to achieve our ends

"Fundamental research" is a matter of perspective. If a developer does not know Rails, he must perform fundamental research into building a new blog engine. While the unknowns, and variability, are significantly smaller, they are still unknown until he does the research. If she were to trod off the beaten path, even slightly, he could run into a community known, but developer unknown, roadblock/design flaw that didn't show up in basic research. (ex. A Rails blog engine vs a multi-threaded Rails blog engine)

Re: Poor developer estimates, a.k.a. guesstimation

#65

Post by jacques_chester »

jared314 wrote: post #64
jacques_chester wrote: post #59

Earlier quoted context omitted.

> But how long does it take to climb a never-climbed mountain? This estimate could be derived in a number of ways. By analogy to other mountains, by creating a statistical model of climbing time based on parameters like gradient and climber skill, or by aggregating expert judgements of the mountain's likely climbing time. A better example of a difficult-to-estimate task was the Manhattan Project. How long would it ta…

> but we are almost never performing fundamental research to achieve our ends "Fundamental research" is a matter of perspective. If a developer does not know Rails, he must perform fundamental research into building a new blog engine. While the unknowns, and variability, are significantly smaller, they are still unknown until he does the research. If she were to trod off the beaten path, even slightly, he could run i…

While all estimates are relative to the context, I disagree that fundamental research is. Fundamental research is about discovering or inventing something nobody has ever done before.

What will be discovered or invented is an unknown; it may be unknown that it is unknown. Research is basically an open-ended search process. There's no foreseable end-state to base an estimate on -- though there are terminating conditions we can use to end research (ran out of money, made amazing discovery that will be passed to engineers to commercialise, gathered sufficient data to write publishable paper etc).

Take the linked post's example of AIDS research. How long will it take to cure AIDS? There is simply no way to know. We can produce estimates, but because it is a research problem, any such estimate would have to have wildly broad ranges to be accurate ("somewhere between 10 and 100 years").

In cases of pure research, you can't estimate the end-goal. Until it's over you may not even know that there were new end-goals you were unaware of (eg. 3M's Post It notes).

Re: Poor developer estimates, a.k.a. guesstimation

#66

Post by jacques_chester »

JoeAltmaier wrote: post #63
jacques_chester wrote: post #30

Earlier quoted context omitted.

All estimates are wrong, what matters is improving their business value by making them more accurate. That some estimates will be less accurate due to uncertainty is not by itself a reason to totally abandon estimating. The generalised case of your argument is that estimation is imperfect in all cases, therefore, we must abandon estimation. This is known as the Nirvana Fallacy: "we have a partial, imperfect solution.…

Wasn't talking about uncertainty at all, so I'm not sure why that comment. And no, its not really cool to pretend I said something else so you can debunk that. I'll repeat myself then: the task of diving into a problem to estimate its time is, for many problems, approximately equal to the time to solve the problem. And I honestly have a hard time figuring out a responsible approach in those cases.

> Wasn't talking about uncertainty at all, so I'm not sure why that comment.

Because you wrote:

> in software projects the country is not just unknown but potentially unknowable

Which is uncertainty.

> I'll repeat myself then: the task of diving into a problem to estimate its time is, for many problems, approximately equal to the time to solve the problem.

Non points-scoring questions: does that happen to you often? Can you give an example? How did you know that the time to estimate would be equivalent to time to implement?

I agree that estimates themselves have a cost/benefit ratio and that sometimes it is going to be negative.

Re: Poor developer estimates, a.k.a. guesstimation

#67

Post by jqjester »

jacques_chester wrote: post #56
x0x0 wrote: post #53

Earlier quoted context omitted.

that "solution" is, basically, ridiculous and not adopted for good reason. A quote from Michael O'Church I quite like: Let's say that you have 20 tasks. Each involves rolling a 10-sided die. If it's a 1 through 8, wait that number of minutes. If it's a 9, wait 15 minutes. If it's a 10, wait an hour. How long is this string of tasks going to take? Summing the median time expectancy, we get a sum 110 minutes, because t…

> The point is, you are unable to know the worst possible outcome. Since it will have huge input into your average, your average is therefore useless. Unbounded worst cases don't happen on every project.

They don't have to be unbounded to be destructive to this estimation method, just large and unknowable.

Re: Poor developer estimates, a.k.a. guesstimation

#68

Post by jacques_chester »

jqjester wrote: post #67
jacques_chester wrote: post #56

Earlier quoted context omitted.

> The point is, you are unable to know the worst possible outcome. Since it will have huge input into your average, your average is therefore useless. Unbounded worst cases don't happen on every project.

They don't have to be unbounded to be destructive to this estimation method, just large and unknowable.

All estimates, in theory, have large and unknowable corner cases. Your problem may require the invention of a new algorithm. Your office may burn to the ground. An asteroid the size of Rhode Island could strike the planet. And so on.

All estimates are uncertain because there is no knowledge of the future. This is true of every profession, of every project.

When we make estimates we have to accept that they are uncertain and that the degree of uncertainty is governed partly by the problem domain.

Re: Poor developer estimates, a.k.a. guesstimation

#69

Post by chrisbennet »

jacques_chester wrote: post #10
onion2k wrote: post #2

This is actually a solved problem in some industries using a technique called "3 point analysis". I've no idea why it hasn't really caught on among developers; it's very useful. Essentially it goes like this: 1. Estimate the very best possible outcome. How long would something take if everything went to plan? 2. Estimate the very worst possible outcome. If everything that can go wrong does, how long will the task tak…

You've gotten to the first step, which is to provide 3 estimates. The next step is to break estimates into smaller parts, estimate them individually, then roll it back up to build a pseudo-statistical profile of probable outcomes. This is the PERT 3-point estimation technique. It's taught in every project management course in the world. It so happens that almost nobody uses it in practice, because it is a pain in the…

How long did you estimate this tool would take to write? :-)
Post reply on HN