What do you mean ‘we need more time’? Project schedule estimation in software

Scenario: Finished requirements, analysis, design; completed development and testing; in UAT; Production deployment: 1 week away.

Customer: Hey this field name does not exactly represent what we want to convey. Can we change its name quickly before we go live?

Eager Novice (who is unfortunately in front of the customer at that moment): Oh yeah, that's just a small change in the Customer model. I think we an get this ready and unit tested in 2 hours.

Customer: Great. Lets git er done!

Project Manager (wary of customer and afraid to rock the boat on an already shaky project): Sure we can do that.

Grizzled Veteran/Grey Beard (who for some reason wasn't consulted on the change request): Hmmm....have we done an impact analysis of the change? What are the downstream impacts on code that references this field? What do we need to do to propagate the change to the db? What about the seven or so Web Services that are impacted by this field change? How long does it take to do a regression test? etc...

Net effect: 2 Hrs initial estimate given to customer; 4 Days worth of work in actuality; Production deployment moved down a week..

> Most can't tell their manager that it will take 2 weeks to parse a CSV because the source data is shit and they have too many other things they are working on. So they stay an extra 6 hours in the office and get called a rock-star until the day that they opt to go to their child's school play instead.

Ugh! For real! I had this happen to me recently. I have trouble telling people "No", and pushy PMs who also happen to be my friends means that I often bite off more than I can chew, and have to either "be a rockstar" (which you start to realize isn't such a compliment), or deliver late / fail to deliver.

So I just started being a dick about it, and it works for me. If I say no too politely, it's really easy to convince me to change to a yes. My turning point was when discussing a project that was much more complex than management assumed it would be (I'm compelled to link the relevant xkcd [0]), and I already felt overwhelmed on my current projects.

The discussion was something like "Ok, how about by this date? What about by this date? What if we dropped these features?", I thought for a moment and responded "Ok, look. I can promise you any date you want, and we can shake hands and leave this meeting. But I'm telling you the reality is I can't do it, and it won't be done. If you'd like to take some of my other projects and move them into the will not be done category, we can talk. I'm only saying this because I don't want to agree to a deadline and then be constantly thinking 'Oh well, that's the date I'm getting fired.'"

I'm very lucky that everyone here is really nice, and realized how stressed I must have been to react like that. So we've reprioritized some things and are beginning to outsource some of my more menial tasks that took up a lot of time.

Didn't really intend for that story to have a point, just venting something my non-dev friends don't relate to. I guess the point could be that the problem isn't just caused by bad managers, or bad dev estimates, or bad process, it's a little bit of everything, and the ratios are different for every company or person.

[0] http://xkcd.com/1425/

Think of it this way - a physician can prescribe a medication and tell you how long you'll be on it, but can she estimate how long it will take before your blood pressure falls within normal parameters? She can tell you what has happened in the past, but suppose you take the medication and your blood pressure remains stubbornly high. Has she failed to meet her targets?

A lawyer may be able to tell you, roughly, how long litigation tends to take, but can your lawyer estimate how long a case will take, what it will cost, what the outcome will be, and the date it will all be wrapped up? If that doesn't happen, did the lawyer fail to meet targets?

Well, in both cases, sort of, yeah. You didn't get the outcome you wanted. But these professions, run by the people who staff them, understand what can be predicted and what can't.

This blog entry uses the task of estimating painting a room as an example, and focuses on breaking the task into subsections. Ultimately, I think the entire approach is deeply flawed, and it all starts by thinking that software development is similar to something like painting a room, just a more complicated version of it.

In my opinion, it's very different. Of course, it's very different from prescribing medication or litigating as well - in fact, I'd say that it's generally considerably less predictable than even these things. I'd say it's closer to writing a novel, creating a film, or recording an album, except that software needs to work, and can be much more mundane at times as well. Ultimately, I don't think the analogies are useful, we need to realize that software is its own thing, and steer clear of too much reasoning by analogy or example.

Unfortunately, the anxiety caused by making estimates under conditions of uncertainty (and possibly duress) is a source of stress and unhappiness for developers, and I do think it's one of the things that drives people out of the field. I actually think this may also be one of the reasons people love creating apps as a hobby but hate doing it for money.