Live data from Hacker News

Why software projects take longer than you think – a statistical model

erikbern.com

21–30 of 324 posts

Re: Why software projects take longer than you think – a statistical model

#21
post #7

Earlier quoted context omitted.

An approach that gives very similar results to yours but is more scientific is to multiply the initial estimates by e (2.718). Or, if you're conservative, then by pi.

The above approach uses the factor 2.2. Why would your factors be any better?

Nitpick: x * 2 * 1.2 == x * 2.4, not 2.2

Re: Why software projects take longer than you think – a statistical model

#22
post #4

I believe this is the reason why scrum uses story points instead of time estimate. By putting uncertainty on the same level as effort, you give it more weight. And using a fibonacci sequence rather than a continuous amount with the rule you should round up if unsure tend to correct those defects.

Yeah in my experience everyone just mentally translates to time anyway so it is pointless. "11 points, that's about 2 days right?"

If you want to make the uncertainty clear it's better to allow ranges, like 2h-5d instead of trying to play stupid mind games.

Story points are like setting your watch 5 mins early so you are on time to things. It doesn't work.

Re: Why software projects take longer than you think – a statistical model

#23
post #7

Earlier quoted context omitted.

An approach that gives very similar results to yours but is more scientific is to multiply the initial estimates by e (2.718). Or, if you're conservative, then by pi.

The above approach uses the factor 2.2. Why would your factors be any better?

Wouldn't it be 2.4 as you are adding 20% of the already doubled estimate?

Re: Why software projects take longer than you think – a statistical model

#24
Honestly, after doing this whole data science thing for a while now, I'm going to be blunt: I can estimate quite a lot of tasks with quite a lot of accuracy. including software and IT tasks.

What I can't do is make bad management hear what they don't want to hear. Nor can I stop people from accepting an estimate because its closer to what they want to be true, because they've already made a promise that conflicts with the actual estimate, or because a certain process requires or necessitates inaccurate estimates.

I think the whole "software is hard to estimate" myth stems from 2 fundamental causes:

- not controlling for human biases or referencing actual real world data - processes that don't punish/ reward people who provide inaccurate/accurate estimates respectively.

Re: Why software projects take longer than you think – a statistical model

#25
post #12

Earlier quoted context omitted.

manager: "thanks for the optimistic estimate!"

Sure, in the same way a bad manager will say "No, that's too high, I'm going to reduce your estimate" if you use a single number. Bad managers are a thing. Make sure you get a good one.

Yep. I still remember my first manager, who doubled every estimate I gave him. He was great.

Re: Why software projects take longer than you think – a statistical model

#26
post #4

I believe this is the reason why scrum uses story points instead of time estimate. By putting uncertainty on the same level as effort, you give it more weight. And using a fibonacci sequence rather than a continuous amount with the rule you should round up if unsure tend to correct those defects.

Yeah in my experience everyone just mentally translates to time anyway so it is pointless. "11 points, that's about 2 days right?" If you want to make the uncertainty clear it's better to allow ranges, like 2h-5d instead of trying to play stupid mind games. Story points are like setting your watch 5 mins early so you are on time to things. It doesn't work.

That happens indeed, but the point (!) stands: the fibonacci scale ensures estimates are rounded up, and that small abstraction merges time and complexity into a single number, whereas it feels more difficult to bump a time estimate based on unknowns/complexity.

Re: Why software projects take longer than you think – a statistical model

#27
post #7

Earlier quoted context omitted.

An approach that gives very similar results to yours but is more scientific is to multiply the initial estimates by e (2.718). Or, if you're conservative, then by pi.

The above approach uses the factor 2.2. Why would your factors be any better?

2.2 is just an arbitrary number.

e is also just an arbitrary number, but it looks like someone calculated it using sophisticated math.

Re: Why software projects take longer than you think – a statistical model

#28
post #4

I believe this is the reason why scrum uses story points instead of time estimate. By putting uncertainty on the same level as effort, you give it more weight. And using a fibonacci sequence rather than a continuous amount with the rule you should round up if unsure tend to correct those defects.

there is an uncharitable response to this thread, which is marked dead. I do think there is a kernel of truth in that response, which is nothing more than mocking.

why fibonacci? I think it's reasonable to say time includes an exponential error. an estimated 1 hour task is very different than an estimated 1 week task. i see that 1 hour, 1 day and 1 week estimates are progressively, and likely exponentially, worse.

Is it just the ease of doing the math? (totally reasonable answer, in my humble opinion). or is there something specific about fibonacci that's actually relevant? I think it's the former not the latter. but if you have any evidence to the contrary, i'd love to hear it.

Re: Why software projects take longer than you think – a statistical model

#29
An important aspect of being a professional software engineer is having the backbone to sometimes say things like:

- “I don’t know yet enough about the problem to give you even a rough estimate. If you’d like, I can take a day to dig into it and then report back.”

- “This first part should take 2-3 days. 5 on the outside. But the second part relies heavily on an API whose documentation and error messages are in Chinese and Google Translate isn’t good enough. I’d need to insist on professional translation in order to even estimate the second part.”

- “The problem is tracking down a bug rather than building something, so I don’t have a good way of estimating this. However, I can timebox my investigation and if I’ve not found the cause at the end of the timebox, can work on a plan to work around the bug.”

You need to be willing to endure the discomfort of looking someone in the face, saying “I don’t know”, and then standing your ground when they pruessure you to lie to them. They probably don’t want you to lie, but there is a small chance that they pruessure you to. If you don’t resist this pruessure, you can end up continually giving estimates that are 10x off-target, blowing past them as you lose credibility, and your running your brain ragged with sleep-deprivation against a problem you haven’t given it the time to break down and understand.

But when you advocate clearly for your needs as a professional, people are generally reasonable.

Re: Why software projects take longer than you think – a statistical model

#30
post #4

I believe this is the reason why scrum uses story points instead of time estimate. By putting uncertainty on the same level as effort, you give it more weight. And using a fibonacci sequence rather than a continuous amount with the rule you should round up if unsure tend to correct those defects.

Yeah in my experience everyone just mentally translates to time anyway so it is pointless. "11 points, that's about 2 days right?" If you want to make the uncertainty clear it's better to allow ranges, like 2h-5d instead of trying to play stupid mind games. Story points are like setting your watch 5 mins early so you are on time to things. It doesn't work.

That's what happens, its all tranlated to hours and cost instantly. I have seen that at several places during poker sessions. The fun starts when management hears this and starts to calculate with it.
Post reply on HN