Software estimation is hard – do it anyway

"Software estimation" is not a specialized problem. Rather, it's just an example of what Daniel Kahneman calls "the planning fallacy." The same phenomena we see with software occur in many, many other fields.

Read "Thinking, Fast and Slow" where he talks about estimating the time to create a new textbook.

> One major “secret” to advancing in a technical career is learning how to give accurate estimates

If Google, Microsoft, Apple, Blizzard, etc can't produce accurate estimates despite employing 'the best of the best', wouldn't that imply it's a nearly impossible task? I can see getting order-of-magnitude estimates, but nothing more accurate than that.

My point is, everyone calls their predictions "goals" and then if they do not reach those, it is for some reason, but external to themselves. It's normal to overfultil or underperform goals, this happens all the time. No one hits every goal 100% every time ("60% of the time, it works every time"), I guess no goal in your company is hit a 100%, it's either over or below.

Marketing has a goal of 5000 new customers. They do not estimate it by conversion history, outreach, customer preferences and other values in their model. But they could call it "estimate" with some thinking. But then they'd have the same problems of being "bad estimators" as developers are.

We in technology call our goals "estimations", and if we're "wrong" we are nailed for it. They are tied to our professional skill in a way goals never are.

Let's call our estimates goals as everyone else does.

Also: Many people confuse estimations and measurements. It's easy to sum 5 throws with a dice. I can do that hundreds of times correctly. It's impossible to predict the sum of 5 throws before the dice is thrown correctly all the time (only on average).

At the risk of being downvoted, I would argue that working on a project without first estimating the work is not engineering. It's coding.

A thoughtful estimate shows care in understanding the project and how it fits into the greater whole of the existing platform. It allows all members of the company to trust in the timelines of the engineering team and align their work to meet the milestones.

An estimate is by no means certain. The size of a project and its novelty will affect the certainty of the estimate. However, not doing one is careless

I remember going to a conference in the 1980s (MacHack), and attending a "Software Project Estimation" workshop.

The guy basically listed excuses for padding the estimate.

Steve McConnell wrote a book about it, using a much more rigorous scientific methodology[0]. He has also written some other stuff about it[1].

This one is really the big one:

"9. Both estimation and control are needed to achieve predictability. "

In my experience, we can accurately estimate software projects that have iron-fisted control. No deviation from the plan. If we use quality-first techniques, like TDD, we can do a fairly good job of hitting targets.

Also in my experience, this results in software that no one wants to use. It doesn't crash, ticks off the punchlist, and basically sucks.

I avoid estimates like the plague (a rare luxury, but I can do it). I like to "wander down the garden path, and see what sights there are," so to speak. I call it "paving the bare spots."[2]

It results in software that comes very close to the user/stakeholder "sweet spot," with great quality. It also tends to come together fairly quickly, and allows for excellent early project visibility.

But that won't work, beyond a fairly humble scope.

[0] https://www.amazon.com/Software-Estimation-Demystifying-Deve...

[1] https://stevemcconnell.com/17-theses-software-estimation/

[2] https://littlegreenviper.com/miscellany/the-road-most-travel...