Why Development Teams Struggle to Deliver on Time, on Budget, or at All • Hacker News Board

Re: Why Development Teams Struggle to Deliver on Time, on Budget, or at All

#161

Post by makmanalp » Tue, Jun 05, 2018, 6:23 PM UTC

Retric wrote: post #136

makmanalp wrote: post #135

Earlier quoted context omitted.

> you simply cannot know all the little details This is the reason big design up front hasn't been the gold standard in planning. Most modern methodologies acknowledge that there'll be a lot of changing requirements and learning-while-building. The point is, given this information, you could a) run around like crazy doing all manner of reckless and crazy cowboy programming and putting out fires b) carefully plan a re…

This is often taken to an unhealthy extreme. You need to have some fairly specific goal in mind to start doing anything. AKA are we building a PS4 game or a self driving car? Methodologies like Scrum are all about transitioning a long term objective into short term progress. But, you really do need some fairly well thought out goal. Further course corrections quickly get very expensive. Sort of like sailing from Virg…

Agreed! Nothing I disagree with here. But last minute itinerary changes from France to Australia back to France again point to a larger, more systematic problems IMHO. Perhaps it's a lack of understanding of the market and customers, or just bad management and lack of company vision - but no software process is going to fix that.

Re: Why Development Teams Struggle to Deliver on Time, on Budget, or at All

#162

Post by house9-2 » Tue, Jun 05, 2018, 6:25 PM UTC

wwarner wrote: post #41

Yes, plan, the more planning the better. But set the deadline first , and plan and design around meeting the deadline. Ask yourself, "I have two weeks to deliver this, but if I had to deliver something tomorrow afternoon, what would I do?" -- and do that first.

A.K.A. deadline driven development

this can work really well if you pare down the list of features and build the simplest version of these features

Re: Why Development Teams Struggle to Deliver on Time, on Budget, or at All

#163

Post by ItsMe000001 » Tue, Jun 05, 2018, 6:29 PM UTC

fvdessen wrote: post #25

kareemm wrote: post #7

In my experience talking to hundreds of software engineering managers, running a software project management business, and being involved on software dev teams for 20+ years, there are two reasons why teams don’t ship or ship poor software. 1. Poor engineering planning. The way to do this well is to break down a feature until they’re at about half-day sizes tasks. Identify any obvious risks or ambiguities with the te…

> The way to do this well is to break down a feature until they’re at about half-day sizes tasks. I have never seen that work in practice.

When I read that in OPs comment I immediately had to think of "How to draw an owl" (the meme version, not one of the serious online lessons -- https://pbs.twimg.com/media/DcC1YEyVQAA9Lkw.jpg). "You simply break the task down into all its parts, then you just do it. Projects are easy."

Not to mention that that approach only works if you produce another version of something you already produced many times. There certainly are such development efforts, for example, I think anything data- and form based, like data entry + database + some reporting, what used to be paper cards a long time ago, and then punch cars, those type of apps can probably be made using that method.

So that's when they had to build the freight version of the Boeing 747. Then they thought "that was easy", and started the Dreamliner, something very different that they had not done before... (and it even still is an airplane, so not even that outrageously different as different pieces of software can be)

I think the "break it down" approach also faces the problem of the "coastline length measurement".

I'm also reminded of an anecdote from my early days with computers: I had to write an assembler routine (as part of an 8 bit assembler program) that seemed too hard, so I outsourced it to someone else. He didn't know what to do either, so I explained it to him. When I was done I realized I had just implemented that difficult function - I wrote it down in half an hour and that was that. Planning well, finding out what to do, is harder than the actual programming. Again and again, in all projects, I find that the >90% part is finding out what to do. There are stories from projects that lost all their code, years of work, and they thought they were done and finished (business kaputt) - but when they rewrote the software they fond it took them only weeks or instead of years. When you already know what you want and how the end result is supposed to look like (and feel and behave) then writing it in code is the easy part.

Re: Why Development Teams Struggle to Deliver on Time, on Budget, or at All

#164

Post by CM30 » Tue, Jun 05, 2018, 6:35 PM UTC

I thought it was because people seem to like changing their product plans midway through development. Seen an awful lot of scope and feature creep in project management, usually either because of a 'client' who keeps wanting more or design by committee.

Though I guess that often ties into project management and planning failures too. Seen a few projects where no one seemingly asked what the client/customer/business needed or how they actually worked, which then meant a ton of refactoring further down the line.

Re: Why Development Teams Struggle to Deliver on Time, on Budget, or at All

#165

Post by markkanof » Tue, Jun 05, 2018, 6:53 PM UTC

gonzo41 wrote: post #16

jhall1468 wrote: post #3

In my experience, deadlines set by higher-ups long removed from (or never having) the technical chops to be determining the deadlines in the first place. VP's and Directors are often the ones dictating the direction, which is great, but then also introducing deadlines, with helpful input from Directors who ALSO haven't touched code in many years. Generally that results in one of two things: * Product delivered on-tim…

Software is up there as the most complex thing ever created. People have been putting up houses for thousands of years, And modern big bridges for hundreds. Big software is ~40-60 years old. Its hard to have rules of thumb and best practice that will deliver the 'on time outcome'. Give it another 100 years. Most dev work right now is still just attempting to deliver correct software. let alone on time or on budget.

In addition, as requirements get more complex and users expect more polished products, we end up relying on more and more open source/reusable infrastructure and libraries. Those tools allow us to not reinvent the wheel and accomplish more in the same amount of time, but they also add to the complexity of any software system.

House building has become somewhat more complicated over the years as building codes are made stricter and new techniques are developed, but for the most part it is just a switch to a new way of doing things vs. in the software world where we will continue to pile on more and more levels of abstraction, but can't reasonably expect to have at least some kind of understanding of the underlying layers.

Re: Why Development Teams Struggle to Deliver on Time, on Budget, or at All

#166

Post by williamdclt » Tue, Jun 05, 2018, 7:23 PM UTC

fvdessen wrote: post #25

kareemm wrote: post #7

In my experience talking to hundreds of software engineering managers, running a software project management business, and being involved on software dev teams for 20+ years, there are two reasons why teams don’t ship or ship poor software. 1. Poor engineering planning. The way to do this well is to break down a feature until they’re at about half-day sizes tasks. Identify any obvious risks or ambiguities with the te…

> The way to do this well is to break down a feature until they’re at about half-day sizes tasks. I have never seen that work in practice.

I'll go against all the other answers and say that I have seen that in practice. I'm even doing that in practice, every day of the week. A half-day task would even be a big task, most of our tickets are a couple hours maximum.

We practice a lean-laced Scrum with 1-week sprints. And yes, it works. And yes, we go fast. So yes, it can work (thought we have several years of refining our methodology and have a big culture of pragmatism and problem-solving).

Re: Why Development Teams Struggle to Deliver on Time, on Budget, or at All

#167

Post by mannykannot » Tue, Jun 05, 2018, 7:39 PM UTC

Aaargh20318 wrote: post #43

The reason that planning software development is bullshit is that you simply cannot know all the little details because in software development you're always doing something you've never done before (because if you did you could just copy-paste your previous work). To use the 'going to the supermarket for milk' example. I could make a fairly accurate estimate for that because I've gone to the supermarket hundreds of…

This claim of inescapable novelty has a kernel of truth, but it is being exaggerated beyond any sense of proportion. If everything you have done in your day-to-day work has been beyond-estimating original, then either you could out-Knuth Knuth (in which case, kudos to you, and I hope you will have time to write the books someday, but your experience is not generally applicable), or you have been goofing off some of the time.

That which is passed off as novelty is often ignorance (not necessarily on the part of the developer.) To continue the retail analogy, software projects are often like the guy who is building a deck on his house, but keeps going back to the hardware store because he made no attempt to think things through, or measure before cutting (note that, in this analogy, the would-be builder is not necessarily playing the role of the developer. In some cases, that might be the hardware store owner's role.)

Re: Why Development Teams Struggle to Deliver on Time, on Budget, or at All

#168

Post by gmiller123456 » Tue, Jun 05, 2018, 7:42 PM UTC

I think software gets a bad wrap when it comes to budget and time overruns. I think we see a lot more software projects go over budget, over time, or fail mainly because they're more common than any other type of complex project.

You don't have to look far for a failed project of another type. While not necessarily a "project", 50% of all businesses completely fail within 5 years [1]. How many Kickstarters have you seen deliver on time? Even if you limit the criteria just to experienced people working in their field? One I've been following [2] is just a book that was supposedly already complete before it was funded was supposed to start in June 2017, and ship in Aug 2017. It's now June 2018 and it still hasn't shipped, that's 600% over time and counting.

I'm not claiming that's scientific proof software estimation isn't worse than anything else. But most of all project's estimation is done behind closed doors, so we'll never get a good feel for how bad/good things really are.

[1] https://fitsmallbusiness.com/small-business-statistics/ [2] https://www.kickstarter.com/projects/pighixxx/abc-basic-conn...

Re: Why Development Teams Struggle to Deliver on Time, on Budget, or at All

#169

Post by thom » Tue, Jun 05, 2018, 7:48 PM UTC

mannykannot wrote: post #167

Aaargh20318 wrote: post #43

The reason that planning software development is bullshit is that you simply cannot know all the little details because in software development you're always doing something you've never done before (because if you did you could just copy-paste your previous work). To use the 'going to the supermarket for milk' example. I could make a fairly accurate estimate for that because I've gone to the supermarket hundreds of…

This claim of inescapable novelty has a kernel of truth, but it is being exaggerated beyond any sense of proportion. If everything you have done in your day-to-day work has been beyond-estimating original, then either you could out-Knuth Knuth (in which case, kudos to you, and I hope you will have time to write the books someday, but your experience is not generally applicable), or you have been goofing off some of t…

But it's exactly this 'ignorance' that is the major part of every software project. There are unknown unknowns, and there is no way from a standing start to grasp either their number or their nature. I'm happy if people are saying that this exploratory work should be done (in part or in total) before estimating, but in my experience that's just not palatable to most organisations.

I envy _hugely_ anyone who builds software that is even 50% the same as what they were working on six months ago.

Re: Why Development Teams Struggle to Deliver on Time, on Budget, or at All

#170

Post by mannykannot » Tue, Jun 05, 2018, 7:51 PM UTC

thom wrote: post #169

mannykannot wrote: post #167

Earlier quoted context omitted.

This claim of inescapable novelty has a kernel of truth, but it is being exaggerated beyond any sense of proportion. If everything you have done in your day-to-day work has been beyond-estimating original, then either you could out-Knuth Knuth (in which case, kudos to you, and I hope you will have time to write the books someday, but your experience is not generally applicable), or you have been goofing off some of t…

But it's exactly this 'ignorance' that is the major part of every software project. There are unknown unknowns, and there is no way from a standing start to grasp either their number or their nature. I'm happy if people are saying that this exploratory work should be done (in part or in total) before estimating, but in my experience that's just not palatable to most organisations. I envy _hugely_ anyone who builds so…

Well, then let's call it what it is, because there are things that can be done to mitigate ignorance to some extent, if there is the will to do so from all parties.