Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why software projects take longer than you think – a statistical model

erikbern.com

Post reply on HN
Print view View original

81–90 of 324 posts

Re: Why software projects take longer than you think – a statistical model

#81

Post by teddyh »

tonyedgecombe wrote: post #74
teddyh wrote: post #60

According to Joel Spolsky¹, programmers are generally bad at estimating, but they are consistently bad, with the exact factor depending on the individual. So by measuring each person’s estimate and comparing it to the actual time takes after the fact, you can determine each person’s estimation factor, and then when they estimate again, you can get a pretty reliable figure. 1. https://www.joelonsoftware.com/2007/10/26…

You know that article was written to sell a feature in their bug tracker. I like Joel's writing but I'd take that piece with a pinch of salt.

If we try to be a bit charitable, we could assume that they implemented the feature in the bug tracker because of this observed property of estimates.

Re: Why software projects take longer than you think – a statistical model

#82

Post by jon-wood »

onion2k wrote: post #6

This is why I like 3 point estimation[1] - if you have optimistic, expected and pessimistic estimates for each task you can pull out which points are high risk. Using a single estimate can't give you that insight. [1] https://en.wikipedia.org/wiki/Three-point_estimation

I’ve had a lot of success using this method, but it does take having management that understand how software development works. A really good PM will look at an estimate like this as a range, with work complete somewhere between the lower and upper bounds. A bad one will take the optimistic estimate as gospel and build everything else around that.

Re: Why software projects take longer than you think – a statistical model

#83

Post by jon-wood »

iraldir wrote: post #4

I believe this is the reason why scrum uses story points instead of time estimate. By putting uncertainty on the same level as effort, you give it more weight. And using a fibonacci sequence rather than a continuous amount with the rule you should round up if unsure tend to correct those defects.

I push back pretty hard on story points whenever I see them. They’re not useful until several iterations in, once you have a baseline for how many points can be delivered in an iteration, and even then one developer’s 1 point story is another’s 5 point.

When estimating I tend to use a coarse scale of hours, days, weeks, and (in extreme cases) months. All estimates then get turned into a range between the minimum and maximum the estimate could mean - for example hours becomes 1 hour to 8 hours, days becomes 1 day to 5 days, and so on.

Anything estimated as weeks or months probably needs scoping until it can be broken down into tasks that will take between an hour and a week.

I find this method works well because it’s intuitive, comes out with a suitable amount of uncertainty relative to the size of the task, and finally results in estimates with units that are useful to project managers and the wider business in scheduling other work needed to get a feature into production.

Re: Why software projects take longer than you think – a statistical model

#84

Post by bonoboTP »

koonsolo wrote: post #53
undergrowth54 wrote: post #29

An important aspect of being a professional software engineer is having the backbone to sometimes say things like: - “I don’t know yet enough about the problem to give you even a rough estimate. If you’d like, I can take a day to dig into it and then report back.” - “This first part should take 2-3 days. 5 on the outside. But the second part relies heavily on an API whose documentation and error messages are in Chine…

I once had a manager/CEO who thought you can negotiate the planning, the same as like a price. "The customer always wants to get it earlier, the engineer later". Me saying that only the developer is able to give a realistic planning, was probably received as "developer wants to push the deadline back in his negotiation". In the end, his company went bust because he was selling stuff to customers that could never be d…

Isn't it the same in price negotiation? Only the producer knows the costs. Perhaps the difference is that money can be shifted around more flexibly. Even if I come out with a negative money balance from a negotiation (costs more than the buyer pays), it might be worth it for long term goals. But you cannot borrow time. You cannot do the 1 week project in -1 weeks and then compensate by doing the next 1 week project in 3 weeks and break even.

Re: Why software projects take longer than you think – a statistical model

#85

Post by TheOtherHobbes »

skrebbel wrote: post #43
leethargo wrote: post #17

Earlier quoted context omitted.

The above approach uses the factor 2.2. Why would your factors be any better?

Because e is more scientific.

Semi-seriously, e is a number defined by recursive growth.

So if you estimate to Level 1, and then break down to Level 2 sub-tasks, then Level 3, your estimate will be out by the same amount each time. and your total error will be a multiplicative series.

If you have an infinite number of levels - which is reasonable for most real-world projects, although this may not be obvious in the planning stages - e will get you nicely in the ballpark.

Re: Why software projects take longer than you think – a statistical model

#86

Post by golergka »

koonsolo wrote: post #59
undergrowth54 wrote: post #29

An important aspect of being a professional software engineer is having the backbone to sometimes say things like: - “I don’t know yet enough about the problem to give you even a rough estimate. If you’d like, I can take a day to dig into it and then report back.” - “This first part should take 2-3 days. 5 on the outside. But the second part relies heavily on an API whose documentation and error messages are in Chine…

> They probably don’t want you to lie, but there is a small chance that they pruessure you to. In my experience, most of the time there was pressure. I remember a funny conversation between my project manager and a colleague: PM: "We need to have X as soon as possible, how long will it take you?" Dev: "Oh that's, easy, I can finish it tomorrow." PM: "Tomorrow??? That's impossible, because it needs a and b, no?" Dev:…

> Does it all take that long?

That's just a question. Asking questions is OK. If you have such a high need to please other people and such a low self-confidence that you will change your answer simply if someone asks you twice, that's the problem with you, not them. I say it as someone who have been like that and have changed it through therapy.

I was a nervous mess and always thought that people around me don't like me, pressure me and in general was very nervous around any interaction where I and other person have some disagreement. But turns out, firmly saying "No" is quite OK, people react well to it, respect you for it and nobody is going to fire you for giving good estimates.

Re: Why software projects take longer than you think – a statistical model

#88

Post by Chris2048 »

bonoboTP wrote: post #84
koonsolo wrote: post #53

Earlier quoted context omitted.

I once had a manager/CEO who thought you can negotiate the planning, the same as like a price. "The customer always wants to get it earlier, the engineer later". Me saying that only the developer is able to give a realistic planning, was probably received as "developer wants to push the deadline back in his negotiation". In the end, his company went bust because he was selling stuff to customers that could never be d…

Isn't it the same in price negotiation? Only the producer knows the costs. Perhaps the difference is that money can be shifted around more flexibly. Even if I come out with a negative money balance from a negotiation (costs more than the buyer pays), it might be worth it for long term goals. But you cannot borrow time. You cannot do the 1 week project in -1 weeks and then compensate by doing the next 1 week project i…

A sales negotiation is pretty zero sum. There is little advantage in paying more for the same product.

However, Giving a developer more time will often improve the quality of the output.

Re: Why software projects take longer than you think – a statistical model

#89

Post by chiefalchemist »

The why is typically:

1) The person(s) doing the estimate aren't qualified to do so.

2) There is a disconnect between wishful thinking and reality.

3) There is some arbitrary future milestone (i.e., "We need to ship by ____ because ____ is happening the following week.") that is independent of software development.

4) Most importantly, when the deadline is missed the estimate / estimators are not questioned, the software team is.

I've been at this a long time - too long? - and the narrative that IT is __always__ at fault is a myth that needs to be buried.

Re: Why software projects take longer than you think – a statistical model

#90

Post by cs02rm0 »

undergrowth54 wrote: post #29

An important aspect of being a professional software engineer is having the backbone to sometimes say things like: - “I don’t know yet enough about the problem to give you even a rough estimate. If you’d like, I can take a day to dig into it and then report back.” - “This first part should take 2-3 days. 5 on the outside. But the second part relies heavily on an API whose documentation and error messages are in Chine…

I find you have to do this, but it can damage professional relationships.

Recently I've flat out said no, as gently as I could, to doing some work - porting a software stack to run on Windows.

A customer is asking for an estimate to do it, because a project manager who wouldn't be developing or supporting (or even using) the system likes Windows. The sales manager isn't thrilled with me over it.

So he's asked the team as a whole to estimate it because I wouldn't. The rest of the guys are frontend devs. I don't have a clue, they're in a worse position but for some reason they're all up for putting a number to it so I look like a bit of a !

I'm a contractor, so I can and would just turn down the work, but it's difficult politically.

Post reply on HN