Why Development Teams Struggle to Deliver on Time, on Budget, or at All

One statistic that has stuck with me is that 80% of software projects fail. It's the default that your software will not succeed, or do well.

One small reason for that is poor time estimation, but having good time estimation won't make your software succeed. I think having good planning, makes for a good working environment, but it doesn't mean the project as a whole will succeed. It might mean the developers will be happier to be with you and pivot to the next idea.

I would like to look at the percentage of successful projects and see what proportion of these had good estimation attributes.

Upon reading the first couple of paragraphs I decided to test it on myself. I had to buy X, which is available in convenience stores (comvini, Japan). I estimated it'd take 5-7 minutes, but I was very surprised when it actually took 4 minutes! To be fair, this has to do a lot more with Japan than with myself. The same test in my home country (Spain) I would have had to guess 15+-5 min during day time. Sidenote: I was very surprised about the concept of "driving for milk", which I'm assuming is a very American thing.

I'm fairly realistic about software estimation which only came after a LOT of retrospection. It normally takes what I estimate, both personally and professionally. The hardest factor I've learned to include is the level of detail. For instance with my personal website [1] I gave myself a full Saturday because I knew I wanted high detail level but I had in mind the overall design. It took the Saturday +1h of a couple of improvements/bugfixes (under 10% error). With my current job I'm also under 10% error.

In the past I have been bitten a LOT about my unrealistic estimations, so the only solution I had to move forward was to learn from those and so I did. So now, from the article, I know that my "quick thinking" is around 50% of the project. I force myself to think a bit more and the details trickle down.

Another thing I've learned is that projects tend to fill as much time as possible (Parkinson's law [2]). So if you are told a deadline, half it! Put the half as your internal deadline, then the project will be just on time.

Finally, complexities are exponential, so learn to say no to unnecessary cruft. "A small change" might seem like a 1% change for business and for you initially, but it will more likely than not grow into a 10-20% change in the end. FFS that is why the duck was added in the first place, to avoid wasting time [3].

[1] https://francisco.io

[2] https://en.wikipedia.org/wiki/Parkinson%27s_law

[3] https://rachelbythebay.com/w/2013/06/05/duck/

I agree. Saying, "Break it down until it's in half-day size tasks" is fine. Some of the things that might interfere:

1. Are you sure you can do that in half a day?

2. You forgot about this one part of the task that you thought would take half a day.

3. There was a bug in the plugin that you were going to use to do this task in half a day, and you spent three days fixing it.

4. This task will actually take longer, but management won't buy that. Looks like it should be half a day on paper, so that's what we'll say.

5. I've never even done this one thing before, but it sounds like it should just take a couple of hours.

I understand the concept - trying to say something will take a week is even more error prone. But I think most of us are still pretty bad at estimating a budget for tasks even at such a small scale.

The thought occurs to me that no matter how much planning you do or how good at estimating you are, the development is going to take however long it's going to take.

You can plan your trip to the store to buy milk and figure out exactly how long it will take, but the only thing that actually matters is that you arrive back home, with milk.

If that milk is absolutely necessary, whether it takes 30 minutes or ten minutes to get it is really a secondary concern. If you spend five minutes getting a better estimate you've delayed the milk by five minutes regardless of how long it takes or how right you were about the timing.

I think we spend too much time thinking about time estimation when the planning we should be doing is figuring out what is so important that the time it takes to build is worth it even if the time estimates are off.

In the context of a fixed-time, fixed-price/budget project it will fail to meet these constraints unless the scope is very clearly delineated at the outset, and ruthless scope/change management is applied throughout the progress of the project. And if at some point someone realizes that there was an omission in the scope definition, the time and budget must change.

If the client or management want to change the scope, in any way, the discussion must immediately start with questions such as what other feature are we dropping, how much time are we adding to the delivery plan, what are the consequences on other features or the current design and code?

Even deciding to remove a feature can cause delays (e.g. having to re-structure a test scenario). Or changing a string in the user interface, which may require re-arranging the layout, re-translating the string, suddenly having to manage plurals, checking that a variation of the old string wasn't already used elsewhere, etc.