Why software projects take longer than you think: a statistical model (2019) • Hacker News Board

Re: Why software projects take longer than you think: a statistical model (2019)

#31

Post by lmilcin » Tue, Mar 09, 2021, 1:07 AM UTC

jbay808 wrote: post #16

> A reasonable model for the “blowup factor” (actual time divided by estimated time) would be something like a log-normal distribution. Interestingly, we did extensive time tracking on a multi-year in-house software project and collected data comparing the estimated completion time of tickets with their actual time. The software department was under a lot of pressure to improve their forecasting, and were somewhat de…

My understanding is that the "multiplicative factor" can typically be mostly attributed to what I call "quality of organization". Not to be confused with efficiency, the quality of organization is measure of how smoothly things go. For example, how likely it is that you are going to be distracted from your task, how much unpredictable synchronous coordination you need to complete your task, etc. An organization which can plan more of its work will have higher quality and an organization where more of its work is of unplanned nature will have lower quality.

There are some possible exceptions. For example, some organizations may have a lot of work unplanned but that unplanned work is segregated to not affect people working on planned work, etc. For example, you may have operations and development teams where operation absorbs most of the unplanned stuff and only rarely bothers the development.

Now, just because work can be accurately planned doesn't mean the organization is efficient. As an example, military tends to be relatively quite good at planning, very inefficiently (ignore the latest failed high profile project...) On the other hand startups tend to be more efficient while also less predictable environments.

I like to think that at some scale of the project some level of trust in projections is necessary to be able to even organize the project. So some drop in efficiency is acceptable in order to be able to plan work.

Re: Why software projects take longer than you think: a statistical model (2019)

#32

Post by mtm7 » Tue, Mar 09, 2021, 1:08 AM UTC

lambda_obrien wrote: post #25

danaliv wrote: post #17

Earlier quoted context omitted.

Totally. Software estimates are like looking at a dungeon map where you can see the beginning and the end, but the rest of the page is blank until you've actually entered that part of the map.

Sometimes you even fall through the floor and find a whole 'nother world down there.

Sometimes you find out that it's a Spirit Temple project and you won't be able to complete it for another seven years.

Re: Why software projects take longer than you think: a statistical model (2019)

#33

Post by balfirevic » Tue, Mar 09, 2021, 1:11 AM UTC

FpUser wrote: post #24

FpUser wrote: post #9

Over the course of my life I've designed and built many products/projects. Not a single one that had deadlines imposed by the customer was late. It might deviate on non critical feature set but the main course has been always served properly.

I know, I'm supposed to be modest but it is the truth. And I did not mean just me personally. Some things I did on my own and for some I had to involve team. I think one of the reasons is that while I am reasonably good at architecture and programming in general I do not really get hung up on processes / languages / tools / technologies etc. What makes me tick is when I see the product working and serving people. Tha…

> I know, I'm supposed to be modest but it is the truth.

I don't think you were downvoted because you were immodest (although I'm sure it didn't help) but because your comment doesn't contribute much to the discussion.

Do you have any comments on the content of the article? Do you have any estimation techniques to share? What were the deadlines based on?

Re: Why software projects take longer than you think: a statistical model (2019)

#34

Post by ChrisMarshallNY » Tue, Mar 09, 2021, 1:15 AM UTC

I think there are a number of things that contribute to problematical estimation, and a number of development tactics that make the damage exponentially worse.

In my experience, I've found that managers will pretty much always end up using Waterfall, the higher you go. It may be agile in the trenches, but it's Victoria Falls, in the boardroom.

At least, projects that have multiple, converging workflows tend to end up looking a bit "waterfallish." I have not had the luxury of working on a standalone, single-line project (until now, as I'm working on my own).

Working on my own, I have developed a number of techniques that start from the assumption that any estimate made, that looks forward more than just a few iterations, is a dumpster fire. I've learned to work around evolving specifications, project progress variances, and the need to reduce tech debt.

This reminds me of some of the work Watts Humphrey did, way in the day, and also a lot of McConnell's work. I like McConnell. He has a very realistic approach. Humphrey's work was too "eggheadish" for me, but people that used his process raved about it.

Re: Why software projects take longer than you think: a statistical model (2019)

#35

Post by jbay808 » Tue, Mar 09, 2021, 1:22 AM UTC

lmilcin wrote: post #31

jbay808 wrote: post #16

> A reasonable model for the “blowup factor” (actual time divided by estimated time) would be something like a log-normal distribution. Interestingly, we did extensive time tracking on a multi-year in-house software project and collected data comparing the estimated completion time of tickets with their actual time. The software department was under a lot of pressure to improve their forecasting, and were somewhat de…

My understanding is that the "multiplicative factor" can typically be mostly attributed to what I call "quality of organization". Not to be confused with efficiency, the quality of organization is measure of how smoothly things go. For example, how likely it is that you are going to be distracted from your task, how much unpredictable synchronous coordination you need to complete your task, etc. An organization which…

In this case, the time tracking was done in a very fine-grained way; you'd log your working time against a given ticket or other thing. So if you got interrupted to go to a meeting or something, that time got logged to the meeting, not the ticket.

Of course interruptions do incur a cost to context switch. But that probably won't explain how one bug fix that you thought would take an hour actually did, and another that you also thought would take an hour took five, especially if you did them both on the same day.

Re: Why software projects take longer than you think: a statistical model (2019)

#36

Post by BigJono » Tue, Mar 09, 2021, 1:26 AM UTC

danaliv wrote: post #17

csours wrote: post #13

Estimates are for projects where you know what you're doing. You can estimate home construction because you've built a home before. I can estimate the happy path. I can't estimate edge and corner cases until I get closer to the edge or corner. It's not even there until I get close to it. I strongly agree with these statements from the article: >"Tasks with the most uncertainty (rather the biggest size) can often domi…

Totally. Software estimates are like looking at a dungeon map where you can see the beginning and the end, but the rest of the page is blank until you've actually entered that part of the map.

What I never get about estimates is why people don't just reduce the granularity until they have some amount of accuracy they can work with.

95% of my teams have estimated using the 'story point' crap, or rarely in days (one in hours for a hot minute, that was a fucking joke I'll tell you what).

In every single one of those teams eventually some stakeholder chucks a hissy fit because some feature they're emotionally invested in was 3 days worth of points and it ended up taking a month.

If they'd measured in weeks, that blowout would have been lost in the noise of other things that took 2 weeks instead of 1, or 3 instead of 2. If they'd measured in months, it wouldn't have been a blowout at all.

Which sounds like putting a band-aid over a gaping wound, but the thing is, what does your business really need to respond to on the time scale of days? The point isn't to improve the estimates (everyone should know by now that that's a fools errand), it's to improve the business response to what the development team is doing.

If you're trying to decide whether to launch in 8 months or 12, or you need to time a project to some external event that's 6 months away, and your plan is to have a big "how are we tracking" meeting once a month, or once a fortnight. Then why do you need to measure in anything more granular than a month, or a fortnight?

Set goals to the cadence of those meetings, try to make them as objective as possible, then hands off and go find something else to do. IME 95% of what teams (especially in enterprise) call estimates is just something for a PO or PM to jack themselves off to for 30 hours a week on projects where their input is only really needed for around 10.

Re: Why software projects take longer than you think: a statistical model (2019)

#37

Post by edoceo » Tue, Mar 09, 2021, 1:48 AM UTC

milesvp wrote: post #23

jbay808 wrote: post #16

> A reasonable model for the “blowup factor” (actual time divided by estimated time) would be something like a log-normal distribution. Interestingly, we did extensive time tracking on a multi-year in-house software project and collected data comparing the estimated completion time of tickets with their actual time. The software department was under a lot of pressure to improve their forecasting, and were somewhat de…

I find this interesting. I've been told that law schools teach that when estimating how long something will take (not how many hours of effort, but calendar time) they say to double your estimate, then increase the units by one. So what you think is a 2 hour task will actually take 4 days to complete. 4 weeks will take 8 months. I wonder how much of that is professional wisdom that embodies this lognormal distributio…

This points to another issue I've observed is that managers/operators ask for an effort-estimate and then map that directly to calendar estimate.

That and thinking that two 4h efforts can be completed in one 8h workday. MFer interrupt work so frequently how do they forget it happens? Forgot you had me in that BS 2h sales meeting "for show"!?

Re: Why software projects take longer than you think: a statistical model (2019)

#38

Post by lostcolony » Tue, Mar 09, 2021, 1:48 AM UTC

FpUser wrote: post #9

Over the course of my life I've designed and built many products/projects. Not a single one that had deadlines imposed by the customer was late. It might deviate on non critical feature set but the main course has been always served properly.

And?

I suspect customers that asked you to, say, build a new inventory tracking tool that interfaces with their existing enterprise infrastructure (SAP, Salesforce, etc) didn't ask you to do it in 2 weeks.

So what was asked was probably -reasonable-. If it wasn't, you probably would have dropped the customer/left the company out of frustration.

If it was reasonable, for a sufficiently fully featured solution that you COULD drop features and still have an MVP, then I would fully expect you to do so.

That's PM Triangle 101; you can give on features while keeping cost and time fixed.

That's just how we handle the fact that estimates are wrong, but we have to keep a date (we drop features, or bring on more people if the work is sufficiently parallelizable). This article is about the estimates being wrong.

Re: Why software projects take longer than you think: a statistical model (2019)

#39

Post by edoceo » Tue, Mar 09, 2021, 1:50 AM UTC

danaliv wrote: post #17

csours wrote: post #13

Estimates are for projects where you know what you're doing. You can estimate home construction because you've built a home before. I can estimate the happy path. I can't estimate edge and corner cases until I get closer to the edge or corner. It's not even there until I get close to it. I strongly agree with these statements from the article: >"Tasks with the most uncertainty (rather the biggest size) can often domi…

Totally. Software estimates are like looking at a dungeon map where you can see the beginning and the end, but the rest of the page is blank until you've actually entered that part of the map.

Fog of War in Starcraft

Re: Why software projects take longer than you think: a statistical model (2019)

#40

Post by csours » Tue, Mar 09, 2021, 2:01 AM UTC

BigJono wrote: post #36

danaliv wrote: post #17

Earlier quoted context omitted.

Totally. Software estimates are like looking at a dungeon map where you can see the beginning and the end, but the rest of the page is blank until you've actually entered that part of the map.

What I never get about estimates is why people don't just reduce the granularity until they have some amount of accuracy they can work with. 95% of my teams have estimated using the 'story point' crap, or rarely in days (one in hours for a hot minute, that was a fucking joke I'll tell you what). In every single one of those teams eventually some stakeholder chucks a hissy fit because some feature they're emotionally…

> "What I never get about estimates is why people don't just reduce the granularity until they have some amount of accuracy they can work with."

I think you've kind of answered yourself here. Software development is fractal-like. The closer you get, the more complicated. If you want to estimate smaller chunks, you have to do a lot more work. At some point it's better to just start developing than to zoom in further, because at that point you're basically developing anyway.