Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why software projects take longer than you think – a statistical model

erikbern.com

Post reply on HN
Print view View original

161–170 of 324 posts

Re: Why software projects take longer than you think – a statistical model

#161

Post by adrianmonk »

One strategy for dealing with risk is to order your tasks so the highly variable ones come first. That way, as you progress through the project, you eliminate a lot of unpredictability, and it's smoother sailing toward the end.

However, as the tables in this article show so clearly, one task can dominate others. This serves as a great illustration of a perception problem the above strategy can create. Outside observers watch your progress and will probably evaluate it in terms of number of tasks completed.

You have (say) 10 tasks to complete, and from their point of view, all they know is a really long time has passed and you haven't even completed 1 of them! Since they are further removed from the situation than you are, they are almost guaranteed to not appreciate why the variable task is that way. They're likely to just think your team is performing badly.

So, maybe it's better to order tasks so that risky stuff is spread more evenly across the timeline of the project. It could create less confusion. Putting risk at the beginning is a strategy that requires a whole lot of trust and buy-in.

Re: Why software projects take longer than you think – a statistical model

#162

Post by adrianmonk »

yawz wrote: post #156

Even the language is not correct: We all call it "estimate", but stakeholders behave like it's a "commitment". Passage from uncertainty to certainty happens in the language, and all the responsibility is on the engineering team's shoulders.

Even if you choose the right words, people don't necessarily pay close attention. And if they want a commitment, they may assume that the numbers you give them are a commitment regardless of how you phrase it.

But even if they did listen closely to what you said, "estimate" is not even a great word. If your car is in a wreck, a body shop gives you an estimate to fix it, and they may treat that estimate as a commitment. It's pretty common practice that people are held to an estimate, or to not going over by a certain small percentage. Maybe we need a word like "forecast" or "prediction".

Re: Why software projects take longer than you think – a statistical model

#163

Post by philk10 »

afarrell wrote: post #111
mgkimsal wrote: post #100

Earlier quoted context omitted.

Testing will happen. The choice you make is who does the testing and when it happens. Internal people testing earlier is generally preferable to external customers testing 'live', but either way, the testing will happen.

“We can’t make those UX improvements to our core open-source product because then we’s have to introduce untested code. The current code is tested because users have used it and reported the bugs and we’ve fixed them.” Is a real thing a project lead said to me at a previous employer.

"Or customers are better at finding the bugs than we are so just give it a quick look over and ship it" - genuine quote from CEO at a company I soon left

Re: Why software projects take longer than you think – a statistical model

#165

Post by TheHypnotist »

mcherm wrote: post #141
Aeolun wrote: post #79

Earlier quoted context omitted.

> But when you advocate clearly for your needs as a professional, people are generally reasonable. This has not been my experience. People want ‘estimates’ at all costs, tell you to not worry about any accuracy, and then a week later tell your manager you committed to x date.

> People want ‘estimates’ at all costs, tell you to not worry about any accuracy, and then a week later tell your manager you committed to x date. I like to handle this problem by providing an estimate RANGE.

Typically when we need something budgetary from a vendor we use the +/- 50% rule. That shouldn't be hard not to screw up.

Re: Why software projects take longer than you think – a statistical model

#166

Post by agumonkey »

undergrowth54 wrote: post #29

An important aspect of being a professional software engineer is having the backbone to sometimes say things like: - “I don’t know yet enough about the problem to give you even a rough estimate. If you’d like, I can take a day to dig into it and then report back.” - “This first part should take 2-3 days. 5 on the outside. But the second part relies heavily on an API whose documentation and error messages are in Chine…

I wish school taught me how to do your first point.

People talk about project management, gantt diagrams and all that but you have no idea how to estimate even grossly the complexity of a thing.

Re: Why software projects take longer than you think – a statistical model

#167

Post by diiq »

Vistimo.com, the tool I built and use when I run estimates for my clients, uses log-normals and monte-carlo simulation to conquer this exact problem.

Really exciting to see other people beginning to recognize and use the same statistical tools -- they've served me really well.

Re: Why software projects take longer than you think – a statistical model

#168

Post by taherchhabra »

undergrowth54 wrote: post #29

An important aspect of being a professional software engineer is having the backbone to sometimes say things like: - “I don’t know yet enough about the problem to give you even a rough estimate. If you’d like, I can take a day to dig into it and then report back.” - “This first part should take 2-3 days. 5 on the outside. But the second part relies heavily on an API whose documentation and error messages are in Chine…

once my CEO asked: "can we add more developers and finish the project faster". I used this famous quote "9 mothers cannot deliver a baby in one month" and we had a big laugh

Re: Why software projects take longer than you think – a statistical model

#169

Post by maltalex »

acd wrote: post #18

US navy has developed something similar with beta statistical distribution. You estimate "Optimistic", "Most likely" and "Pessimistic" time estimates for each task in the project and then use beta distribution on it. Some tasks take way longer than estimated. Here is the link to the time estimation described above with Beta distribution. https://www.isixsigma.com/methodology/project-management/bet...

I find this approach very interesting, but it hinges on the assumption that project completion times follow a beta distribution. What's the basis for that?

Re: Why software projects take longer than you think – a statistical model

#170

Post by achenatx »

The assumption in the article is the amount of work is a constant. Instead what happens is that a manager has decided how much time it should take (always too little). If you accept that time, then you will go over.

If you push for more time and get it, then the manager will eventually add more scope and you will go over.

The vast majority of the time we see projects go over deadlines because scope was added under the guise of clarifying scope.

Starting with a business value actually works pretty well to enable people to control scope.

We also use actual velocity on tasks to re-forecast daily. That has been the most effective way to get a good date. Over a large number of tasks this works very well.

Post reply on HN