Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why software projects take longer than you think – a statistical model

erikbern.com

Post reply on HN
Print view View original

251–260 of 324 posts

Re: Why software projects take longer than you think – a statistical model

#251

Post by fdjlasdfjl »

afarrell wrote: post #247
commandlinefan wrote: post #158

Earlier quoted context omitted.

> Negotiation about scope. I used to work for an online travel site - before every project kicked off, the business analysts would produce requirements documents that always started with a "flexibility matrix". The flexibility matrix had three rows labeled "schedule", "scope", and "resources" and three columns: "flexible", "moderately flexible" and "inflexible" and they were supposed to check one option for each row.…

Wait, how are resources flexible here? You can’t just scale the number of engineers on a project quickly..

Depends on the number of devs you have working on the project. If you only have 1 then you can flex to 4 pretty easily and still have an efficient team. If you have 7 and flex to 10 then you're just adding inefficiency.

Re: Why software projects take longer than you think – a statistical model

#252

Post by Bjartr »

adrianmonk wrote: post #162
yawz wrote: post #156

Even the language is not correct: We all call it "estimate", but stakeholders behave like it's a "commitment". Passage from uncertainty to certainty happens in the language, and all the responsibility is on the engineering team's shoulders.

Even if you choose the right words, people don't necessarily pay close attention. And if they want a commitment, they may assume that the numbers you give them are a commitment regardless of how you phrase it. But even if they did listen closely to what you said, "estimate" is not even a great word. If your car is in a wreck, a body shop gives you an estimate to fix it, and they may treat that estimate as a commitmen…

> Maybe we need a word like "forecast" or "prediction".

Scrum (The org behind that flavor of agile) actually changed the language used in their guide back in 2011 from developers making a "spring commitment" to making a "spring forecast" for exactly these reasons.

https://www.scrum.org/resources/commitment-vs-forecast

Re: Why software projects take longer than you think – a statistical model

#253

Post by JabavuAdams »

My empirically-confirmed heuristic is that the time to deliver a feature set that someone would actually want to use is 2.5x-3x of the time I think of when asked for an off-the-cuff estimate.

Basically, multiply initial estimate by a number between e and pi -- no joke! It's a bit of a problem given that PM's think they're being generous with a 20% pad.

Re: Why software projects take longer than you think – a statistical model

#254

Post by SkyBelow »

golergka wrote: post #86
koonsolo wrote: post #59

Earlier quoted context omitted.

> They probably don’t want you to lie, but there is a small chance that they pruessure you to. In my experience, most of the time there was pressure. I remember a funny conversation between my project manager and a colleague: PM: "We need to have X as soon as possible, how long will it take you?" Dev: "Oh that's, easy, I can finish it tomorrow." PM: "Tomorrow??? That's impossible, because it needs a and b, no?" Dev:…

> Does it all take that long? That's just a question. Asking questions is OK. If you have such a high need to please other people and such a low self-confidence that you will change your answer simply if someone asks you twice, that's the problem with you, not them. I say it as someone who have been like that and have changed it through therapy. I was a nervous mess and always thought that people around me don't like…

>Asking questions is OK.

Is it?

Questions have built in assumptions and implications. Most questions don't have enough to be of concern, but some questions definitely imply things beyond the question itself. When asked 'Does it all take that long?', many hear an implied 'Make it not take so long.' command. It is not implied all the time, but it is implied often enough. This does not mean someone has low self-confidence or needs therapy.

Re: Why software projects take longer than you think – a statistical model

#255

Post by commandlinefan »

Bartweiss wrote: post #233
jerf wrote: post #193

Earlier quoted context omitted.

You sound like you're complaining, but your complaint may be incomplete. If those bullet points were honored, that is indeed how pretty much every organization doing development will operate, whether it likes it or not. That is not intrinsically a problem. I take it these requirements documents were not honored by your non-developers? A document labeling scope as "moderately inflexible" does little if it's treated as…

One basis for complaint that stands out to me is framing "flexible resources" as a full degree of freedom. The first question is of course whether that's true , since resources don't tend to flex up unless someone outside the project dictates it. But even if it's accurate, it tends to produce the situation The Mythical Man-Month was written to condemn. If the ship date can't move and the scope won't bend on core feat…

Actually it started out as a 2x2 matrix (scope/schedule), and all of them were scope = flexible and schedule = inflexible (of course), and they started adding the "resources = flexible" row so that they could walk back their claims of scope being flexible, because of course it wasn't.

Re: Why software projects take longer than you think – a statistical model

#256

Post by ska »

piccolbo wrote: post #231
mikekchar wrote: post #42

The interesting thing is that by the central limit theorem, the mean of a mean is normally distributed. This is extremely helpful. Here's what I suggest you do: Same size your stories to small values. Do 30 stories in a sprint and take the mean. Do 30 sprints and take the mean of the sprint. What you get is the mean amount of time to do a sprint of 30 stories. What's amazing is that this estimate will be normally dis…

The central limit theorem is in the limit for the number of variables in the sum approaching infinity. In the finite world, the article explains how it's done. The article is saying, the sum of lognormals is not normal. You are saying: take enough of them and it is normal. The article is still more accurate than your reasoning for 30 stories. From the wikipedia entry for Central limit theorem " As an approximation fo…

    You are saying: take enough of them and it is normal.
This doesn't completely undermine your point, but that isn't what they are saying, I think. I read it as saying by CLT that the estimates of the mean of those distributions is normal and centered on [the mean you are actually interested in]. Tails are perhaps somewhat a red herring here, because you don't really care about them unless you are specifically trying to evaluate worst-case-but-really-unlikely.

Re: Why software projects take longer than you think – a statistical model

#257

Post by a_c »

undergrowth54 wrote: post #29

An important aspect of being a professional software engineer is having the backbone to sometimes say things like: - “I don’t know yet enough about the problem to give you even a rough estimate. If you’d like, I can take a day to dig into it and then report back.” - “This first part should take 2-3 days. 5 on the outside. But the second part relies heavily on an API whose documentation and error messages are in Chine…

I always consider four factors in estimating software cost

- the scope of the task

- the resource/people you have

- the confidence in the resource/people

- and the dependency on both the resource and the task

Estimation goes wrong in many ways. In my experience, I categorize as follow:

- estimated by non-technical PM, unable to gauge technical/business complexity

- estimated too early in development life cycle. Often time someone come up with idea X then ask right away "shouldn't be difficult, how long will it take"

- no view on team member's productivity due to lack of measurement or measuring the wrong thing

- not considering dependency in the development cycle. e.g. The question "whether our only backend engineering available for the task" is often omitted

- not considering testing, documentation, 3rd party integration/procurement, maintenance and deployment cost

The common practise of coming up a number, then double, or maybe triple it up to make the estimation does not address why estimation is off. Mis-estimation is a symptom. The cause and the cure lies in the people

Re: Why software projects take longer than you think – a statistical model

#259

Post by c0vfefe »

yawz wrote: post #156

Even the language is not correct: We all call it "estimate", but stakeholders behave like it's a "commitment". Passage from uncertainty to certainty happens in the language, and all the responsibility is on the engineering team's shoulders.

> stakeholders behave like it's a "commitment".

Many places treat budgets the same way.

Re: Why software projects take longer than you think – a statistical model

#260

Post by eikenberry »

afarrell wrote: post #108
nahname wrote: post #104

Earlier quoted context omitted.

As long as software is a cost center to your company, it will be treated this way. Trying getting a job at a company where software is a core concern. Ideally with a CEO that is not from a marketing/business background.

For more on this sort if thing, Patrick McKenzie’s writing is good: - https://www.kalzumeus.com/2011/10/28/dont-call-yourself-a-pr... - https://www.kalzumeus.com/2014/04/09/what-heartbleed-can-tea...

> In the real world, picking up a new language takes a few weeks of effort and after 6 to 12 months nobody will ever notice you haven’t been doing that one for your entire career.

Spoken as someone who has never taken the time to fully master a programming language and, from the sound of it, has never worked with someone who has either. The difference between someone who has spent 6-12 months with a language compared someone who has spent 6-12 years is night and day. From the general tone of the article, he obviously focuses more on the business value than on the technical side and that is a pretty good approach for making money. But I'll take Peter Norvig's advice (http://norvig.com/21-days.html) over this guys when it comes to mastering a language.

To be fair most of the content of those articles is pretty decent though and it is just a pet peeve of mine when people claim that mastery of your medium doesn't matter.

Post reply on HN