Software estimation is hard – do it anyway

Oh, wow. Usually I get asked to estimate with very vague descriptions of what I am going to build, with hard pressure on date of delivery. At one point I said to a manager that if he pushes hard enough he will get any estimate he wants. Well, for the scientifically inclined, humans can estimate programming tasks that take around one hour with good precision. Precision declines until one week and anything over a month is virtually impossible to estimate.

One person's estimate is different from another person's estimate. The hole that most developers dig for themselves starts with a fixation on having perfect numbers and that they'll be punished if they don't have these. Estimation is often a part of negotiation separate from commitments, but developers treat it as a pure analysis game.

This results in punishment for poor communication. The punishment starts when they claim they can't provide anything that approaches estimate, or attempt to weasel out of a discussion. It should be clear to the customer when and how you are delivering commitments. Poor communication turns an estimation discussion into one where a developer has overcommitted.

Even if you can assign zero time estimates to tasks, or even zero estimates for how long time estimating task will take, you can provide a view on how you'd go about it and the relative priorities for your investigation. This is a critical part of building trust which is necessary for the long-term success of any project. Communicating a clear perspective that does not make commitments is important for building the relationship which will give you wiggle room late on when you have to make adjustments.

When a commitment is made to the customer, any associated estimate needs to be provided along with context. Providing a confidence number leads to misinterpretation because different tasks may require different analyses. It is better to encode it in some other way to highlight things like: multiple interviews conducted, whether a coding spike was done in the area, whether support contracts are in place for the 3rd party service required, etc.

As the project progresses, there should be some kind of update to those commitments. This is where again it gets scary for people because they don't like having these candid discussions.

In all of this, I don't prescribe any methodology. This can fit with any methodology, but you have to find a way to fit it in. Waterfall has lots of clear points where commitments are made, but it doesn't have the feedback mechanism in its purest form. Updating commitments is essentially part of "agile", but the recording and communication can sometimes be a challenge. The job of the developer is do enough of the right work to set the right commitments and communicate around them.

This is my experience, over and over - to the point where I often get to the point the article states as "just … give up".

Then there's the "let's split it up into pieces first". This is where, instead of rolling one 100-sided die, we flip 100 coins to get a better estimate.

But my absolute favorite is when the Project Manager asks for an estimate and you give a number and if they think it's too high or too low, they will keep asking until you give them the number they were looking for in the first place. Why even ask? Because now it's your fault if it's wrong!

(side note: There are solutions to these things and they are definitely not the right way to do things and are signs of a toxic environment - but there is hope!)

He can ... really really accurate estimate, ... a lot of time .. building .. verifying .. actual requirements.. designing ... solution .. validating .... process will require dev resources.. resources... people.. support team.. a lot of time.. very expensive.. scary.. and I'll be nakedly responsible for my own mistakes..

"How about we do it that other way. You mentioned devs could keep throwing stuff against the wall. I like how that sounds."

> I can give you a really really accurate estimate, but in order to do so we're going to have to spend a lot of time going through the request, building and verifying actual requirements, designing the solution and then validating it.

This is almost surely wrong for most developers, or else rewrites wouldn't fail to deliver within the estimated time so often. Rewrites per definition already has a perfect specification in the old code, just write something working the same way using a new architecture. But that is still really hard to deliver apparently.

Of course it could be true in your case, but you can't blame the inability of software engineers in general to estimate tasks on that.