Why software projects take longer than you think – a statistical model

Explicit is better than implicit. Asking is better than guessing. If you think that someone is implying something, ask them.

In most cases, people who ask "Does it really take that long?" are giving you their first emotional reaction and haven't actually thought it through deep enough to decide to manipulate you. They're upset and they want to make sure that they understood you right - which is a thing people often do when they get bad news, and I cannot blame them: if you get upset about something, at least make sure you really should be upset.

So, you should react either in a spock-like "stupid rational", giving no thought to what they might be implying whatsoever with a simple "yes", or, even better, in an emotionally-aware rational way: acknowledge their emotions and offer options. There are always options and trade-offs, they just need to be clearly communicated. Not "OK, I'll do it in three days instead of five", but "I'll can do it in three days with prototype quality so we can test it out, but it will be buggy as hell, not fit for production, and you should document it". I've never had a manager who had a problem with that, and I despite how popular horror stories are, I really doubt that there's a lot of them out there.

Your manager cares about the product, he's emotionally invested in it and wants everything to go as fast as possible - it's not a bad thing! It's only bad if you view his disappointment as your fault - but it's not. It's just how things are. As a developer, you're often a messenger bringing bad news. There's no way around it, and you have to learn to deliver bad news for people. And separate these bad news from you and your professional self-confidence.

Here is a story about the woes of interpreting statistical distributions:

I have two habits when I estimate: First, I like to give a range or even better a triple estimate of "optimistic", "expected", "worst case". (And btw, you should expect things to weight toward the pessimistic side, because you always find new requirements/problems, but you rarely discover a supposed requirement is not really one.)

Second: I like to break down a project into tasks of just a few hours and then add everything up. I usually don't share that spreadsheet with the customer, but it helps me a lot. Pretty much always it gives me a number higher than I'd like, but which is almost always very accurate. A couple times I've ignored the result and given a lower estimate because I really wanted some project, and it has always turned out the spreadsheet was right.

Well, one time I combined these two approaches, so that my very finely-chopped estimates all had best/expected/worst values, and I shared that with the customer. Of course they took one look at the worst-case total and said, "How can a little thing like this possibly take 2 years??" I didn't get the work. :-)

EDIT: Btw it feels like there is a "coastline paradox" here, where the more finely you estimate, the higher the max possible, so that you can make your estimate grow without bound as long as you keep splitting items into greater detail. It'd be interesting to see the math for that.

Pet theory: this is entirely explained by unknown systems not behaving as expected. As developers, and unlike e.g. carpenters, we are constantly using new tools with effects we haven't yet experienced. Then we have to yak-shave to get around their heretofore unknown kinks. Then the time blows up.

If and when you're using known features of a known framework, and that's all you're doing, the estimates are accurate and the work is completed quickly.

> You need to be willing to endure the discomfort of looking someone in the face, saying “I don’t know”, and then standing your ground when they pruessure you to lie to them.

This. I have PMs who will ask me over and over until I give a number and I've learned to stand my ground. Because if I don't, I end up being responsible for the estimate I've given (as I should).

Now I make it clear that I will not give a number until I know more. Just a few weeks ago I was asked to give an estimate of how long it would take to do an integration with something I've never used before. I said, "I need 1-3 days to learn the product enough to give an estimate". What I get back is "can't you just give WAG". But this time around I said "You have two choices, 1. you give me time to learn what I need and then I'll give an estimate or, 2. Find someone who already knows the product to give you an estimate"

EDIT2: In spite of my personal experience I do think the author makes a strong case for this: "Adding up task estimates is a really misleading picture of how long something will take." Perhaps I've had better results because I try to give myself a little padding in every task, just considering that everything requires not just typing code but getting on some calls, having back-and-forth in emails and the project management tool, testing (automated or manual), fixing a thing or two, etc. So my individual estimates are probably a bit higher than median. When I work with other developers they consistently estimate lower than me. But my numbers are deliberately not "best case", because then you know you'll go over on the total.