Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why software projects take longer than you think: a statistical model (2019)

erikbern.com

Post reply on HN
Print view View original

141–150 of 178 posts

Re: Why software projects take longer than you think: a statistical model (2019)

#141

Post by jjk166 »

This is a simple project, it shouldn't take nearly that long. Let's set the deadline a little earlier and leave the rest as buffer.

This project is ahead of schedule, so you're not going to need the buffer, we'll reallocate resources for that time.

The project is on schedule, let's move up the deadline a little just so we have a bit of buffer at the end.

This project is slightly behind schedule, let's schedule a daily hour long meeting to go over everything to get things back on schedule.

The project is further behind schedule, how about you redo a major part of the project to try a different approach that will hopefully be sufficiently faster that you'll not only recover the cost of the reimplementation but also get back onto the original schedule.

The project has fallen even further behind schedule, and other things are starting to pile up, we need to pull some resources away to deal with them.

The project is ridiculously far behind schedule, people are losing faith in your ability to deliver, you need to get back on schedule before they'll approve more resources for this project.

The project has fallen too far behind schedule, we're pulling the plug. I've scheduled a lengthy post-mortem meeting so we have plenty of time to go over how you failed to bring this project to completion so you will learn from your mistakes.

Re: Why software projects take longer than you think: a statistical model (2019)

#142

Post by JohnFen »

bonoboTP wrote: post #23

All this smokes and mirrors, when the reality is: if you gave accurate estimates, you'd get reprimanded, not get the project approved, etc. Same reason that infrastructure project go over budget. The way anything actually gets done is by initially being overly optimistic, ignoring potential future problems, getting the project approved, then lock it in via sunk costs, now as problems turn up, you can claim it could n…

> the reality is: if you gave accurate estimates, you'd get reprimanded, not get the project approved, etc.

I have never actually experienced this personally, but I have no doubt that it's a thing in many companies.

But it's really inept management. If management has some sort of hard date in mind, then the way they should pose the problem to dev is: we need this completed by xx/xx/xx -- what can be accomplished in that time?

Re: Why software projects take longer than you think: a statistical model (2019)

#143

Post by JohnFen »

sublinear wrote: post #135
bonoboTP wrote: post #23

All this smokes and mirrors, when the reality is: if you gave accurate estimates, you'd get reprimanded, not get the project approved, etc. Same reason that infrastructure project go over budget. The way anything actually gets done is by initially being overly optimistic, ignoring potential future problems, getting the project approved, then lock it in via sunk costs, now as problems turn up, you can claim it could n…

Indeed. Development should not begin until all requirements have been finalized. Every project I've ever been on that had wildly wrong estimates was because management didn't actually plan jack shit either due to inexperience, laziness, malice, or all of the above. They then play the blame game to try to keep their job, and man it's so hilarious to witness the sheer carnage as everyone tears them a new asshole nowada…

> Development should not begin until all requirements have been finalized.

I agree, but this runs directly counter to Agile methodologies which are all the rage these days.

Re: Why software projects take longer than you think: a statistical model (2019)

#144

Post by [deleted] »

pmg102 wrote: post #78
distcs wrote: post #73

Earlier quoted context omitted.

> The reason planning poker exists is to create a sort of prisoner’s dilemma between developers to stop this getting out of hand. Planning poker creating a sort of prisoner's dilemma is an intriguing thought. Mind explaining a little how it leads to something like prisoner's dilemma. I'd like to grok the connection between the two things.

I'll have a go: If you ask developer A how long it will take, the best outcome for them is that they AND EVERYONE ELSE go high. So they should go high. BUT if everyone else goes low, but they go high, then they look bad. So they're forced to go lower. But then not too low, or they won't be able to deliver in time. So perhaps this settles on an estimate that's "as low as possible but no lower"?

[deleted]

Re: Why software projects take longer than you think: a statistical model (2019)

#145

Post by ramses0 »

commandlinefan wrote: post #69
ramses0 wrote: post #43

Conversely: “chicken nugget” estimation. Traditional agile suggests “sizing each task” (work estimated to be less than one sprint). I’ve found that simply “counting each task” is more effective... instead of trying to estimate the difference in sizes, it's easier and more accurate to try to divide the work into similarly sized chunks. There may be "the usual" variation around delivery pace, but at the end it all aver…

This kind of cheerleading is a big hit with the people who consume estimates... not so much with the slobs stuck producing them.

The slobs stuck producing them (aka: myself) then break down a project into:

Project => Area(epic/feature) @ T-Shirt Size => ...you're done...

If you can backtest/backport simply the COUNT of stories associated with enough epic/feature sizes, then you can forward-predict an arbitrary new "@ SIZE" to determine an approximate story-count.

Inputs:

    XL @ [ 25 stories, 33 stories, 52 stories, ...etc... ] => Mean/Median/StdDev
    L @ [ 20 stories, 15 stories, 18 stories, ...etc... ] => Mean/Median/StdDev
    ...etc...
...basically "the slobs producing estimates" are on the hook _MOSTLY_ and _ONLY_ for accurately accounting/tracking/submitting _CURRENT_ work (ie: put your stories in the sprint/board/backlog). The only other responsibility is being _CONSISTENT_ on submitting and tracking mostly "similar sized items" instead of "wildly varying sized items of indeterminate complexity.

If there are "large and wild" areas, they should be bumped up and "promoted" to an epic/feature rather than tracked as a sub-task on an existing epic/feature (talk to your manager/product-owner ;-).

It's relatively straightforward to maintain that "story hierarchy" of "Project => Feature/Epic => Story/Task". It's relatively straightforward to estimate Feature/Epic sizes (S/M/L/XL, with extra attention or avoidance of L/XL, strictly because there tends to be more variation on those sizes).

With relatively straightforward math tracking backwards four quarters (how many stories in total were closed by this team?), breaking down a "new project" or "new work" becomes more an exercise in "is this big or small?" rather than "what precise date or size is this particular piece of work."

For a high confidence interval, commit to _LESS_ Projects/Features than you'd done before (and you have StdDev to guide you on the probable upper and lower bounds), other than that, just prioritize at the Project/Feature level to get the most important or most valuable work scheduled ASAP.

Re: Why software projects take longer than you think: a statistical model (2019)

#146

Post by TheCoelacanth »

_ea1k wrote: post #94
mpweiher wrote: post #30

Earlier quoted context omitted.

> in a meeting-heavy agile environment Everytime I see that, I cringe. But sadly, you're not wrong. Agile is anti-meeting, anti-heavy and definitely anti-meeting-heavy. I really don't know what to call this weird thing that was created. Just as an example: why are there standups? Because in XP, meetings are frowned upon . So the idea is not to have meetings whenever humanly possible (better to pair up, talk to indivi…

Just wait until the Scaled "Agile" Framework[1] experts help to improve your efficiency with even more process overhead. [1] They weren't even concise with the acronym. Hey, lets throw a random e on the end!

Scaled Agile Framework is possibly the least agile thing anyone has ever done.

Re: Why software projects take longer than you think: a statistical model (2019)

#147

Post by aleph_minus_one »

raincole wrote: post #93
bonoboTP wrote: post #23

All this smokes and mirrors, when the reality is: if you gave accurate estimates, you'd get reprimanded, not get the project approved, etc. Same reason that infrastructure project go over budget. The way anything actually gets done is by initially being overly optimistic, ignoring potential future problems, getting the project approved, then lock it in via sunk costs, now as problems turn up, you can claim it could n…

By this logic, programmers should be able to accurately estimate the schedule for personal projects or small scale open source projects where no one can "reprimand" them. It's so far from reality that it's almost satire.

> By this logic, programmers should be able to accurately estimate the schedule for personal projects or small scale open source projects where no one can "reprimand" them.

In my experience many good and experienced programmers are able to do that. The reason why you get different observations is that in many cases, if you do such a project, you do it because the journey is the destination, i.e. you do such projects to try out new interesting technologies, i.e. the central gola is not finishing the project, but to get exposure to programming topics that you consider interesting. Finishing such small scale open-source projects to a given maturity is just rather a desired side effect and not a goal.

Re: Why software projects take longer than you think: a statistical model (2019)

#148

Post by mpweiher »

steveBK123 wrote: post #59
bombolo wrote: post #55

Earlier quoted context omitted.

agile is like the no true Scotsman. There's never a true agile :D

Yeah it cracks me up every time. 90% of people are like "the agile industrial complex is killing my job" and the other 10% are like "well you haven't REALLY done agile because..".

Both of these can be true at the same time.

And in fact ARE true.

The Agile Industrial Complex is a horror show, but that doesn't mean it's not possible to do agile well.

There absolutely is a way of doing agile well, I've been involved in a number of teams that did. And it worked extremely well. In fact, we did agile way before the agile manifesto came out, and before I was aware of there being a name for the things we were doing. We were just doing them because they make sense, they make us go fast and they let us have fun while doing it. Which is basically what the XP people said when they started writing things down. They never claimed to have invented a brand new way of creating software. No, they were observing that certain teams were very productive, and looked at what those teams were doing. If there was a pattern. Spoiler: there was.

Anyway, I also introduced agile "practices" in a large company. Well, in a small team in a large company. We didn't do a single one of the practices the Agile Industrial Complex proposes and often mandates. The much more important team next to us did. They did all the AIC practices. We did the technical. And interacted closely. Did TDD, worked on trunk, paired when necessary, did stuff alone when not. Did the simplest thing that could possible work. ("Where's your database?" "We'll put it in when we need it". "Oh we're done. I guess we didn't need the database ¯\_(ツ)_/¯" ).

The more important team next to us that was doing Scrum with the standups and whathaveyounot failed. We delivered. Hmm.

And if you can point to the practices being promoted by the AIC as being in direct and obvious conflict with, for example, what's written the Agile Manifesto, and can in fact point to ways of doing it right that are in harmony with the AM and in conflict with the AIC, then it ain't a No True Scotsman fallacy. It's a simple case of the AIC doing it wrong.

Re: Why software projects take longer than you think: a statistical model (2019)

#149

Post by magicink81 »

Has anyone tried the Shape Up method by Basecamp? Their concept of “fixed time, variable scope” seems like it might break the Gordion Knot of software estimation challenges.

Their book is available for free online, here's a link to the chapter "Estimates don’t show uncertainty " https://basecamp.com/shapeup/3.4-chapter-13#estimates-dont-s...

Re: Why software projects take longer than you think: a statistical model (2019)

#150

Post by awinter-py »

drittich wrote: post #91
commandlinefan wrote: post #65

Earlier quoted context omitted.

> if you gave accurate estimates, you'd get reprimanded I like the cynicism, but this implies that you can give accurate estimate, which I also take exception to. To give an accurate estimate, you'd have to first have a complete specification, which I've never seen.

Pretty much correct, IMO. Really, the first build (and usually the only one, unfortunately) IS the process of estimating. I try to POC as much as possible to make sure I know where the landmines are. Just this week, I blew by one of my own recent estimates because I assumed I could pretty easily determine the output column names and data types for any arbitrary SQL Server stored procedure using a SQL query. Many impl…

yes this

difficult to scope some projects without starting the build

Post reply on HN