Live data from Hacker News

Why software projects take longer than you think – a statistical model

erikbern.com

181–190 of 324 posts

Re: Why software projects take longer than you think – a statistical model

#181
post #171

Earlier quoted context omitted.

Remember - the pressure is to get you to work unpaid overtime under the (mistaken) belief that more hours = more software.

It's about having "passion"!

oh man this hits close to home

Re: Why software projects take longer than you think – a statistical model

#182
post #176

Pet theory: this is entirely explained by unknown systems not behaving as expected. As developers, and unlike e.g. carpenters, we are constantly using new tools with effects we haven't yet experienced. Then we have to yak-shave to get around their heretofore unknown kinks. Then the time blows up. If and when you're using known features of a known framework, and that's all you're doing, the estimates are accurate and…

I disagree. Estimations tend to be just as wrong even when the tools are well known.

There's always that one edge case you haven't considered, that one algorithm that doesn't work as well as you expected, that small change to the requirements the requires a completely different approach.

Re: Why software projects take longer than you think – a statistical model

#183

In my experience software takes longer to build than original estimates because no one will get out of the way of the development team and let them work. This is an extreme example, but one I now live in daily. My current full-time-ish gig is working on a pretty enterprisy system for law enforcement. To this date there hasn't been a single feature request, or bug fix that took more than 16 hours of development time.…

Sorry but why would a manager even do a code review?

Re: Why software projects take longer than you think – a statistical model

#184
As always, my favorite article on this subject: https://www.lesswrong.com/posts/CPm5LTwHrvBJCa9h5/planning-f...

> A clue to the underlying problem with the planning algorithm was uncovered by Newby-Clark et al., who found that

Asking subjects for their predictions based on realistic “best guess” scenarios; and

Asking subjects for their hoped-for “best case” scenarios . . . . . . produced indistinguishable results.

> So there is a fairly reliable way to fix the planning fallacy, if you’re doing something broadly similar to a reference class of previous projects. Just ask how long similar projects have taken in the past, without considering any of the special properties of this project. Better yet, ask an experienced outsider how long similar projects have taken.

Re: Why software projects take longer than you think – a statistical model

#185

An important aspect of being a professional software engineer is having the backbone to sometimes say things like: - “I don’t know yet enough about the problem to give you even a rough estimate. If you’d like, I can take a day to dig into it and then report back.” - “This first part should take 2-3 days. 5 on the outside. But the second part relies heavily on an API whose documentation and error messages are in Chine…

> You need to be willing to endure the discomfort of looking someone in the face, saying “I don’t know”, and then standing your ground when they pruessure you to lie to them. This. I have PMs who will ask me over and over until I give a number and I've learned to stand my ground. Because if I don't, I end up being responsible for the estimate I've given (as I should). Now I make it clear that I will not give a number…

To anyone else that doesn't know what WAG is, it's apparently "Wild Ass Guess".

Re: Why software projects take longer than you think – a statistical model

#186
post #176

Pet theory: this is entirely explained by unknown systems not behaving as expected. As developers, and unlike e.g. carpenters, we are constantly using new tools with effects we haven't yet experienced. Then we have to yak-shave to get around their heretofore unknown kinks. Then the time blows up. If and when you're using known features of a known framework, and that's all you're doing, the estimates are accurate and…

I disagree. Estimations tend to be just as wrong even when the tools are well known. There's always that one edge case you haven't considered, that one algorithm that doesn't work as well as you expected, that small change to the requirements the requires a completely different approach.

"Just as" wrong? I don't know what to point to to resolve disagreement here since it's just anecdotal, but if you're just using the same feature you've used hundreds of times before, there is nowhere near the potential for yak-shaving snags.

Re: Why software projects take longer than you think – a statistical model

#187
post #183

In my experience software takes longer to build than original estimates because no one will get out of the way of the development team and let them work. This is an extreme example, but one I now live in daily. My current full-time-ish gig is working on a pretty enterprisy system for law enforcement. To this date there hasn't been a single feature request, or bug fix that took more than 16 hours of development time.…

Sorry but why would a manager even do a code review?

He was originally hired as the sole developer 10 years ago, but the project grew too big, and instead of hiring a manager to oversee the project and hire more devs, they moved him into a management position, and put him in charge of hiring new developers.

Re: Why software projects take longer than you think – a statistical model

#188
My theory on why software estimates suck is tied to Rosenberg’s Law:

> Software is easy to make, except when you want it to do something new. The corollary is, The only software that’s worth making is software that does something new.

It's hard to estimate the difficulty of something you've never done. And the nature of software development is to always do new things.

Re: Why software projects take longer than you think – a statistical model

#189

In my experience software takes longer to build than original estimates because no one will get out of the way of the development team and let them work. This is an extreme example, but one I now live in daily. My current full-time-ish gig is working on a pretty enterprisy system for law enforcement. To this date there hasn't been a single feature request, or bug fix that took more than 16 hours of development time.…

I give estimates in hours. They're really by days for large projects and half days for smaller projects, but I express them in hours. I then provide a breakdown of the hours the team has been able to spend to date, as well as the hours that each forced interruption took. That way, when my manager asks why he perceives us as being behind on the project, I can tell him exactly how long his pointless meetings and changing priorities have delayed the team. It's also a good CYA technique for when he tries to blame the team when his manager asks why the project is delayed.

Re: Why software projects take longer than you think – a statistical model

#190
> Instead, figure out which tasks have the highest uncertainty – those tasks are basically going to dominate the mean time to completion.

From the technical side of things, uncertainty can mean a few things here:

(A) I've never done this kind of task (or I don't remember or didn't write down how long this task took in the past)

(B) I don't know how to leverage my historic experience (e.g., implementing an XYZWidget in React and implementing the same widget in Vue or Elm for some reason take different amounts of time)

Considering (A)... Rarely does a seasoned developer in the typical business situation encounter technical tasks that are fundamentally different than what has been encountered before. Even your bleeding-edge business idea using modern JS' + GraphQL' is still going to be built from the same fundamental pieces as your 1999 CRUD app using SOAP and the estimates are going to be the same.

If you disagree with this you are in the (B) camp or you haven't done the work to track your estimates over time and see how ridiculously accurate estimates can be for an experienced practitioner. Even "soft tasks" like "design the widget" are estimable/repeatable.

This whole you-can't-estimate-software accuracy position is entirely a position of inexperience. And of course all bets are off there. You are talking about estimating learning in this case, not doing. And the bets are especially off if you aren't modeling that these are two different activities: learning and doing.

Post reply on HN