Live data from Hacker News

Why software projects take longer than you think: a statistical model (2019)

erikbern.com

41–50 of 178 posts

Re: Why software projects take longer than you think: a statistical model (2019)

#41

All this smokes and mirrors, when the reality is: if you gave accurate estimates, you'd get reprimanded, not get the project approved, etc. Same reason that infrastructure project go over budget. The way anything actually gets done is by initially being overly optimistic, ignoring potential future problems, getting the project approved, then lock it in via sunk costs, now as problems turn up, you can claim it could n…

It's undeniably true that estimates are often driven mostly by what number will be acceptable. But it doesn't invalidate the points the article makes. Even when this distorting element of business drivers are removed, estimation is still very hard.

You might get reprimanded if you give accurate estimates - that doesn't change the fact you mostly can't give accurate estimates even if you wanted to.

Re: Why software projects take longer than you think: a statistical model (2019)

#42

I write code for 15 years and I still have no idea how to do any estimations. All my estimations are off by 2-200 times. Basically they’re useless. I write code until I’m happy with it. Sometimes I need to rewrite code 5 times before I’m satisfied. Sometimes I have extremely bad mood and can’t do anything creative at all, so even expecting me to work a hour tomorrow is not reliable. Or may be I’ll be in perfect mood…

In one of the Agile training courses I took we were taught the concept of "horizon of predictability". The base one is two weeks.

Within two weeks you more or less can estimate accurately. Beyond the two weeks, the estimates become pretty much useless. "Three months" is not an estimate. It just can't be one in good faith.

Re: Why software projects take longer than you think: a statistical model (2019)

#43
Conversely: “chicken nugget” estimation.

Traditional agile suggests “sizing each task” (work estimated to be less than one sprint).

I’ve found that simply “counting each task” is more effective... instead of trying to estimate the difference in sizes, it's easier and more accurate to try to divide the work into similarly sized chunks. There may be "the usual" variation around delivery pace, but at the end it all averages out. Usually 5-10 chicken-nuggets-per-sprint (or 10-20 or whatever your pace is) gives you the "fine grained" velocity or pace that is useful for getting rough estimates in the medium-term.

Extraordinarily large "chicken nuggets" become clearly visible because they "take too long" and that's a conversation with the task owner about trying to keep sizing down, or a clue to break them down or add more people (and consequently- more tasks/stories).

For overall project size estimation, you can/should have more S/M/L/XL concepts, with effectively back-testing of estimated size to task breakdown count, or be able to state "in the last quarter, this team delivered 2XL, 4L, 3M, 8S".

Who knows the true size differential between the 17 delivered (well, maybe if you do further breakdown, and simple task counts you can...), but it's a starting point.

Team size may change, focus may vary, vacation varies, amount of non-productive (maintenance, planning, migration, compliance) work may change, but at least it starts giving you a starting point.

Finally, recognizing that variance should narrow over time. Driving from SF-to-NY may have an initial estimate of "1234", but if you get to Iowa and you're at 800, then you have a much better chance/estimate that the remainder will be another ~800.

Your variance should be smaller because you're estimating a smaller sized item (only half the work), and you're estimating (nay- duplicating!) actual observed delivery pace.

TL-DR: don't size stories, do size "epics/features", consistently track delivery, backtest based on previous data, refine estimates as you get closer, be able to estimate when you're "past" the 50% mark

Re: Why software projects take longer than you think: a statistical model (2019)

#44

I'm genuinely looking for a "calm" company. Is there such a thing? I have a few anecdotal stories of companies being absolutely chaotic (my current one included). I don't know where to point fingers to. I could start at pointing myself. Customers demanding custom features. Execs and sales people asking for unreasonable estimates. Engineers not feeling safe enough to say "no" but have to make something work, introduci…

What you're looking for exists in the government and large universities.

Re: Why software projects take longer than you think: a statistical model (2019)

#45

All this smokes and mirrors, when the reality is: if you gave accurate estimates, you'd get reprimanded, not get the project approved, etc. Same reason that infrastructure project go over budget. The way anything actually gets done is by initially being overly optimistic, ignoring potential future problems, getting the project approved, then lock it in via sunk costs, now as problems turn up, you can claim it could n…

It depends on the company, or more specifically on how the company manages the budget (people working on projects, infrastructure budgets, etc.). In many companies people would rather do the other way around: inflate their estimates to make sure that they definitely fit the reality later. This also often leads to continuously increasing budgets because or the corporate logic: “if you asked for budget but didn't use, then why did you ask for it’? So people would be actively looking for ways to spend any excess, to make sure that next time they will be trusted with their estimate, which in turn would likely be bigger than the previous one. Crazy, but true.

So, it can work both ways really.

Re: Why software projects take longer than you think: a statistical model (2019)

#46

All this smokes and mirrors, when the reality is: if you gave accurate estimates, you'd get reprimanded, not get the project approved, etc. Same reason that infrastructure project go over budget. The way anything actually gets done is by initially being overly optimistic, ignoring potential future problems, getting the project approved, then lock it in via sunk costs, now as problems turn up, you can claim it could n…

It's undeniably true that estimates are often driven mostly by what number will be acceptable. But it doesn't invalidate the points the article makes. Even when this distorting element of business drivers are removed, estimation is still very hard. You might get reprimanded if you give accurate estimates - that doesn't change the fact you mostly can't give accurate estimates even if you wanted to.

> you mostly can't give accurate estimates even if you wanted to.

Ah, an optimist.

I long for a world where software development estimates and those who expect them are perceived as the unfunny jokes they are. Why do naked emperors make such wretched despots?

Re: Why software projects take longer than you think: a statistical model (2019)

#47

All this smokes and mirrors, when the reality is: if you gave accurate estimates, you'd get reprimanded, not get the project approved, etc. Same reason that infrastructure project go over budget. The way anything actually gets done is by initially being overly optimistic, ignoring potential future problems, getting the project approved, then lock it in via sunk costs, now as problems turn up, you can claim it could n…

Shoot for the stars and you might hit the moon.

Re: Why software projects take longer than you think: a statistical model (2019)

#48
post #8

What I find ironic is that everyone in the industry knows that estimating is very hard, thus estimations are nearly useless, yet everyone insists on the importance of planning and on basing such planning on void estimates. Nobody has yet had the balls to state the obvious: we have to learn to work without estimations.

>Nobody has yet had the balls to state the obvious: we have to learn to work without estimations.

Age old conundrum: "Who will bell the cat?"

Re: Why software projects take longer than you think: a statistical model (2019)

#49

Another factor here is in a meeting-heavy agile environment, many engineers can't even give a good forward estimate of the number of "in the zone" hours they will have in the coming week. Will I be on 0 outage calls or 5? Doesn't matter if I am on pager duty rotation or not.. Will I get pulled into 3 calls this afternoon or 0? Is my boss gonna come bother me with some urgent non-ticket task tomorrow morning? Etc. I'm…

Another factor here is in a meeting-heavy agile environment, many engineers can't even give a good forward estimate of the number of "in the zone" hours they will have in the coming week.

As a developer my highest level of productivity was on a 2 person team with a bi-weekly meeting with our manager. We would have impromptu meetings when needed, but for the most part I was free to work 6-15 hours in a row whenever I got in the zone, and if I slacked off other times noone noticed.

Once we got more people and switched to scrum my productivity plummeted. My anxiety went through the roof. Most days the standup left me feeling like I wanted to go back to bed. Most of the time when I was in the zone and feeling engaged I had to waste it on meetings that had no tangible benefits.

Busy work is draining, management knows this, they don't care.

Re: Why software projects take longer than you think: a statistical model (2019)

#50

Another factor here is in a meeting-heavy agile environment, many engineers can't even give a good forward estimate of the number of "in the zone" hours they will have in the coming week. Will I be on 0 outage calls or 5? Doesn't matter if I am on pager duty rotation or not.. Will I get pulled into 3 calls this afternoon or 0? Is my boss gonna come bother me with some urgent non-ticket task tomorrow morning? Etc. I'm…

Another factor here is in a meeting-heavy agile environment, many engineers can't even give a good forward estimate of the number of "in the zone" hours they will have in the coming week. As a developer my highest level of productivity was on a 2 person team with a bi-weekly meeting with our manager. We would have impromptu meetings when needed, but for the most part I was free to work 6-15 hours in a row whenever I…

Yes I self-select for sub-5 member teams now, its such a better life.
Post reply on HN