51–60 of 72 posts
Also be aware of the selection bias when it comes to estimations. The companies A, B and C all put in estimates for a product based on similar premises. Company C gets selected because their estimate is the lowest. This could be because company C is the best and can do the work quicker but it is also possible that company C has just made a low estimate pretty much by chance. Estimates that are wrong (lower than the a…
Earlier quoted context omitted.
You are absolutely right. It's impossible to estimate how long it will take to complete something you have never done before in your life. Unless the new code being written is absolutely trivial, estimates are going to be guesses, at best. The only half-decent solution I found so far is iterative development. Take a feature and break it down i to big rocks. Further break down the big rocks into little chunks - each t…
> Take a feature and break it down i to big rocks. Further break down the big rocks into little chunks - each that you estimate would take you less than a day to complete. Experimental psychologists have found that decomposing tasks increases estimate accuracy, regardless of how or why the task is decomposed. It's called the "unpacking effect". http://www.sciencedirect.com/science/article/pii/S0022103103...
I usually estimate that writing the code is anywhere from 10-20% of completing the project.
Some research shows that 1 kloc of code contains between 15 and 50 defects, and that code reviewing will pick up only 70-90%. This still leaves a lot of bugs. I sometime ago wrote up some meta research on estimating code reviewing: http://specialbrands.net/2011/10/24/code-review-in-scrum-xp-...
This is actually a solved problem in some industries using a technique called "3 point analysis". I've no idea why it hasn't really caught on among developers; it's very useful. Essentially it goes like this: 1. Estimate the very best possible outcome. How long would something take if everything went to plan? 2. Estimate the very worst possible outcome. If everything that can go wrong does, how long will the task tak…
A quote from Michael O'Church I quite like:
Let's say that you have 20 tasks. Each involves rolling a 10-sided die.
If it's a 1 through 8, wait that number of minutes. If it's a 9, wait 15
minutes. If it's a 10, wait an hour.
How long is this string of tasks going to take? Summing the median time
expectancy, we get a sum 110 minutes, because the median time for a task is
5.5 minutes. The actual expected time to completion is 222 minutes, with 5+
hours not being unreasonable if one rolls a lot of 9's and 10's.
This is an obvious example where summing the median expected time for the
tasks is ridiculous, but it's exactly what people do when they compute time
estimates, even though the reality on the field is that the time-cost
distribution has a lot more weight on the right. (That is, it's more common
for a 6-month project to take 8 months than 4. In statistics-wonk terms, the
distribution is log-normal.)
Software estimates are generally computed (implicitly) by summing the
good-case (25th to 50th percentile) times-to-completion, assuming perfect
parallelism with no communication overhead, and with a tendency for
unexpected tasks, undocumented responsibilities, and bugs to be overlooked
outright.
[1]
An example from my background: I work on ml code. A learning algorithm was written in parallel with a particular loss function (essentially, this tells you if your estimates are good or bad). The person writing the spec assumed that a different loss function could be dropped in later. Indeed, this is how the math works on paper. But loss function number 2 required a different data layout between the machines. The vision was writing a new loss function should be one or two lines of code dropped into the middle of a hot loop; the reality was it took almost a month of work.
Many developers don't believe that management is going to listen to their estimates so they either avoid giving an estimate or they tell management what they want to here. A friend of mine worked at a place that was 'trying' agile (at the behest of top management) and had them entering estimates for tickets and recording time for them. My friend heard from his boss that his "estimates were too high" and when he found…
Boss: Give me an estimate on how long it will take to do Project X.
Developer: Six months.
Boss: That's not acceptable! We promised it to the customer in one month.
Developer: OK, one month.
Earlier quoted context omitted.
You've gotten to the first step, which is to provide 3 estimates. The next step is to break estimates into smaller parts, estimate them individually, then roll it back up to build a pseudo-statistical profile of probable outcomes. This is the PERT 3-point estimation technique. It's taught in every project management course in the world. It so happens that almost nobody uses it in practice, because it is a pain in the…
Build your tool as a plug-in for existing popular project management tools like Rally. No one will use yours if they have to enter user stories and tasks in two places. I would love a tool that could analyze best / middle / worst case estimates for tasks plus account for predecessor relationships between user stories and generate a probability distribution chart of possible release dates.
This is actually a solved problem in some industries using a technique called "3 point analysis". I've no idea why it hasn't really caught on among developers; it's very useful. Essentially it goes like this: 1. Estimate the very best possible outcome. How long would something take if everything went to plan? 2. Estimate the very worst possible outcome. If everything that can go wrong does, how long will the task tak…
that "solution" is, basically, ridiculous and not adopted for good reason. A quote from Michael O'Church I quite like: Let's say that you have 20 tasks. Each involves rolling a 10-sided die. If it's a 1 through 8, wait that number of minutes. If it's a 9, wait 15 minutes. If it's a 10, wait an hour. How long is this string of tasks going to take? Summing the median time expectancy, we get a sum 110 minutes, because t…
Unbounded worst cases don't happen on every project.
It was mentioned before in the thread, but I'll just go ahead and leave this here again: http://en.wikipedia.org/wiki/Planning_fallacy Trying to improve developer estimates is just another case of doing the wrong thing, righter.
But it's not. First, psychologists have done research on how to ameliorate it. See the "unpacking effect" paper I linked elsewhere. Kahneman and others have also pushed using "reference classes". In the estimation literature this is called "estimation by analogy" and it is a well-established technique.
Which reveals the next thing about the Planning Fallacy; it comes from the psychological literature. There are at least two other bodies of literature which need to be dealt with before we give up entirely. The first is ordinary project estimation literature; most of the advanced stuff is by people from the Operations Research field. The second is work done by Statisticians working on what they call forecasting which is essentially parametric estimation of timeseries data.
These 3 bodies of literature seem to have evolved largely in isolation. I haven't see OR papers talking about the psych research, I haven't seen the psychologists citing the International Journal of Forecasting and so on.
Earlier quoted context omitted.
You've gotten to the first step, which is to provide 3 estimates. The next step is to break estimates into smaller parts, estimate them individually, then roll it back up to build a pseudo-statistical profile of probable outcomes. This is the PERT 3-point estimation technique. It's taught in every project management course in the world. It so happens that almost nobody uses it in practice, because it is a pain in the…
It's turned out to be a deeper rabbit hole than I expected. That's quite ironic, given the subject of this thread....
One of them is "Why Research Is Almost Impossible To Estimate".
Earlier quoted context omitted.
This is basically your solution: http://imgur.com/YNmKId9 PHB: Use the CRS database to size that market. Dilbert: That data is wrong. PHB: Then use the SIBS database. Dilbert: That data is also wrong. PHB: Can you average them? Dilbert: Sure. I can multiply them too. ----- I kid (sorta). It misses the core issue, why are developers estimates so terrible, 3 terrible estimates don't make a "good" one. The post hits on…
> Estimations are based on experience, yet developers are constantly building brand new types of products. If all you built was "blog engines" over and over again, you would get damned good at estimating them. The trick in software is as soon as something needs to be reused over and over again, it is abstracted, put in a library, built into a framework, so you can make your product different (read: build something en…
This estimate could be derived in a number of ways. By analogy to other mountains, by creating a statistical model of climbing time based on parameters like gradient and climber skill, or by aggregating expert judgements of the mountain's likely climbing time.
A better example of a difficult-to-estimate task was the Manhattan Project. How long would it take to build the first atomic weapon? That was a truly novel problem, insofar as it would require fundamental physics research to solve.
But almost none of our work is of that nature. To be sure, there are unexpected requirements and complexities, but we are almost never performing fundamental research to achieve our ends.
Earlier quoted context omitted.
> Take a feature and break it down i to big rocks. Further break down the big rocks into little chunks - each that you estimate would take you less than a day to complete. Experimental psychologists have found that decomposing tasks increases estimate accuracy, regardless of how or why the task is decomposed. It's called the "unpacking effect". http://www.sciencedirect.com/science/article/pii/S0022103103...
Unpacking works great when it's clear what the tasks are going to be. In my experience the bulk of the work does not go into the project or tool itself, i.e. completing the tasks, but rather into resolving issues. I usually estimate that writing the code is anywhere from 10-20% of completing the project. Some research shows that 1 kloc of code contains between 15 and 50 defects, and that code reviewing will pick up o…
McConnell's book has a really excellent table of items that are left out of estimates. Nuts-and-bolts things like "API documentation", "deployment scripts", "status emails" and so on.
> This still leaves a lot of bugs.
Right. The classical software engineering theory is to build multiple quality gates and to study them for defects yielded. I recall that inspection stomps everything else; automatic testing came second.
The problem of time-to-fix is subtly different from time-to-release, though. Software with known defects is regularly in use.