Poor developer estimates, a.k.a. guesstimation
alexefish.com
Poor developer estimates, a.k.a. guesstimation
1–10 of 72 posts
Re: Poor developer estimates, a.k.a. guesstimation
#2This is actually a solved problem in some industries using a technique called "3 point analysis". I've no idea why it hasn't really caught on among developers; it's very useful. Essentially it goes like this:
1. Estimate the very best possible outcome. How long would something take if everything went to plan?
2. Estimate the very worst possible outcome. If everything that can go wrong does, how long will the task take?
3. Estimate the most likely outcome. Knowing what you do about the client, the task, the tech, etc, how long is it likely to take.
(Point 3 is the normal way of estimating.)
The advantage of 3 point analysis is that you can use the difference between the times to estimate risk. If the best and worst outcomes are similar, then there's little risk of things going horribly wrong. If the best and the likely times are similar but the worst is much longer then there's something that can completely screw up - identify what that is and defend against it. If the likely and worst times are similar but the best time is much shorter, identify why that's the case and try to make sure it happens. And so on.
Better yet, by taking the aggregate best times for a set of requirements and comparing it against the worst, you can get a good view of the overall risk of a project.
Very useful indeed.
Re: Poor developer estimates, a.k.a. guesstimation
#3This is actually a solved problem in some industries using a technique called "3 point analysis". I've no idea why it hasn't really caught on among developers; it's very useful. Essentially it goes like this: 1. Estimate the very best possible outcome. How long would something take if everything went to plan? 2. Estimate the very worst possible outcome. If everything that can go wrong does, how long will the task tak…
This is basically your solution: http://imgur.com/YNmKId9
PHB: Use the CRS database to size that market.
Dilbert: That data is wrong.
PHB: Then use the SIBS database.
Dilbert: That data is also wrong.
PHB: Can you average them?
Dilbert: Sure. I can multiply them too.
-----
I kid (sorta). It misses the core issue, why are developers estimates so terrible, 3 terrible estimates don't make a "good" one.
The post hits on some of the points, but IMHO misses a major one. Estimations are based on experience, yet developers are constantly building brand new types of products. If all you built was "blog engines" over and over again, you would get damned good at estimating them. The trick in software is as soon as something needs to be reused over and over again, it is abstracted, put in a library, built into a framework, so you can make your product different (read: build something entirely new) -- and EVERYONE sucks at estimating the cost of building things entirely new.
Think of the (entirely new in 1962) Lunar Module... they thought it would cost at max 350 million. It cost 2.2 billion. Those where very smart people with a major stake in getting the initial number right, but it was just a crazy ass guess because they had NEVER BUILT ONE BEFORE.
Re: Poor developer estimates, a.k.a. guesstimation
#4Also be aware of the selection bias when it comes to estimations.
The companies A, B and C all put in estimates for a product based on similar premises.
Company C gets selected because their estimate is the lowest.
This could be because company C is the best and can do the work quicker but it is also possible that company C has just made a low estimate pretty much by chance.
Estimates that are wrong (lower than the actual cost) are more likely to be selected so it always looks like estimates are bad even if all the estimates submitted actually do cluster around the true value
Re: Poor developer estimates, a.k.a. guesstimation
#5This is actually a solved problem in some industries using a technique called "3 point analysis". I've no idea why it hasn't really caught on among developers; it's very useful. Essentially it goes like this: 1. Estimate the very best possible outcome. How long would something take if everything went to plan? 2. Estimate the very worst possible outcome. If everything that can go wrong does, how long will the task tak…
This is basically your solution: http://imgur.com/YNmKId9 PHB: Use the CRS database to size that market. Dilbert: That data is wrong. PHB: Then use the SIBS database. Dilbert: That data is also wrong. PHB: Can you average them? Dilbert: Sure. I can multiply them too. ----- I kid (sorta). It misses the core issue, why are developers estimates so terrible, 3 terrible estimates don't make a "good" one. The post hits on…
You are absolutely right. It's impossible to estimate how long it will take to complete something you have never done before in your life. Unless the new code being written is absolutely trivial, estimates are going to be guesses, at best.
The only half-decent solution I found so far is iterative development. Take a feature and break it down i to big rocks. Further break down the big rocks into little chunks - each that you estimate would take you less than a day to complete.
Only after completing your first "big rock" in this way can you use your new data to gauge your velocity and give an estimate for completing the rest of the work that's worth the paper it's written on.
Re: Poor developer estimates, a.k.a. guesstimation
#6This is actually a solved problem in some industries using a technique called "3 point analysis". I've no idea why it hasn't really caught on among developers; it's very useful. Essentially it goes like this: 1. Estimate the very best possible outcome. How long would something take if everything went to plan? 2. Estimate the very worst possible outcome. If everything that can go wrong does, how long will the task tak…
IIRC, 1 & 3 are usually the same, and 2 is usually still way too optimistic. And that's from research, not anecdote.
It still might be useful. Use 1 to anchor 3 to anchor 2 and then triple it.
Re: Poor developer estimates, a.k.a. guesstimation
#7I recently read Software Estimation (Steve McConnell, author of Code Complete). My main takeaway was to distrust developer intuition and instead build estimates from data. One example is to look at the duration of past projects of "similar size" (still requires some qualitative analysis). Another example is to count the number of distinct "features" or "components," and build an estimate from those. Both of these still require some qualitative analysis. That will be the weak point of any estimate. The goal is to minimize inputs from developer intuition and use as much quantitative analysis as possible when estimating.
Re: Poor developer estimates, a.k.a. guesstimation
#8I recently read Software Estimation (Steve McConnell, author of Code Complete). My main takeaway was to distrust developer intuition and instead build estimates from data. One example is to look at the duration of past projects of "similar size" (still requires some qualitative analysis). Another example is to count the number of distinct "features" or "components," and build an estimate from those. Both of these sti…
I don't think Steve McConnell was suggesting that developer intuition was not relevant. In some of the formulas he suggests for estimation he includes a factor for how accurate the team was in estimating previous projects. Ultimately each developer will need to estimate individual features or other small units.
The other big take away from that book is that estimates should not be a single number but rather a range. If the client insists on a fixed price bid then you start with the higher end of your estimated range and add in something for the risk of a fixed price bid.
Re: Poor developer estimates, a.k.a. guesstimation
#9I've always found this article -- "Evidence Based Scheduling" -- by Joel Spolsky to be rather compelling:
http://www.joelonsoftware.com/items/2007/10/26.html
I'm curious if anyone else has tried it?
Re: Poor developer estimates, a.k.a. guesstimation
#10This is actually a solved problem in some industries using a technique called "3 point analysis". I've no idea why it hasn't really caught on among developers; it's very useful. Essentially it goes like this: 1. Estimate the very best possible outcome. How long would something take if everything went to plan? 2. Estimate the very worst possible outcome. If everything that can go wrong does, how long will the task tak…
You've gotten to the first step, which is to provide 3 estimates. The next step is to break estimates into smaller parts, estimate them individually, then roll it back up to build a pseudo-statistical profile of probable outcomes.
This is the PERT 3-point estimation technique. It's taught in every project management course in the world.
It so happens that almost nobody uses it in practice, because it is a pain in the rear end to do in Excel. It also so happens that I'm working on a tool to make it easy[1].
All I have right now is a landing page, proof-of-concept code and some articles. Oh, and a pile of research papers and books. It's turned out to be a deeper rabbit hole than I expected.
edit: how come "professional" is the only word that was spelled wrong on the landing page? Because Murphy is hilarious, that's why.