Live data from Hacker News

Why Development Teams Struggle to Deliver on Time, on Budget, or at All

7pace.com

231–239 of 239 posts

Re: Why Development Teams Struggle to Deliver on Time, on Budget, or at All

#231

The reason that planning software development is bullshit is that you simply cannot know all the little details because in software development you're always doing something you've never done before (because if you did you could just copy-paste your previous work). To use the 'going to the supermarket for milk' example. I could make a fairly accurate estimate for that because I've gone to the supermarket hundreds of…

This claim of inescapable novelty has a kernel of truth, but it is being exaggerated beyond any sense of proportion. If everything you have done in your day-to-day work has been beyond-estimating original, then either you could out-Knuth Knuth (in which case, kudos to you, and I hope you will have time to write the books someday, but your experience is not generally applicable), or you have been goofing off some of t…

I agree.

There is also a lot you can do to decrease impact of novelty on the overall schedule and most people I work with do exactly nothing to reduce variability in the process and improve their estimation ability.

It is easy to claim every time you write code you are doing something new but, hey, every time they build a building they do something new. Yet buildings (except for extremely ambitious projects) typically can be delivered reliably by reputable developers.

Re: Why Development Teams Struggle to Deliver on Time, on Budget, or at All

#232
post #228

Earlier quoted context omitted.

Firstly you you are still assuming the same gross exaggeration of novelty, and secondly, you can do analysis and planning at a much higher level of abstraction than development. Waterfall gave the impression that planning and analysis is costly by insisting it should be carried down to the lowest levels of abstraction, before doing anything else.

Just to give an example of two projects I worked on at the same company within a few months of each other: The first involved taking all sorts of situational data from various sources in an airport, e.g. radar systems with their own binary formats, and combining them with the work of a postgrad researcher who had come up with an algorithm to optimise when pilots should turn on an aircraft's engine in order to save fu…

Aaargh20318 et al are making the claim that it is always inescapably like this and there's nothing you can do about it, because every day of programming is always fundamentally unlike anything you have ever done before. All I am suggesting is that this is an exaggeration.

Doing something about it takes some commitment, however. Nobody estimates well without having practiced it, and been honest with themselves about where it went wrong.

Re: Why Development Teams Struggle to Deliver on Time, on Budget, or at All

#233

Earlier quoted context omitted.

I'll go against all the other answers and say that I have seen that in practice. I'm even doing that in practice, every day of the week. A half-day task would even be a big task, most of our tickets are a couple hours maximum. We practice a lean-laced Scrum with 1-week sprints. And yes, it works. And yes, we go fast. So yes, it can work (thought we have several years of refining our methodology and have a big culture…

Do you mind putting out a number of hours (per dev team) that get spent in Backlog curation, story generation, and other budget planning every week? I think perhaps an important missing ingredient for getting budgets right is actually spending the time to do so. The smaller the company, the harder it seems to justify. Indeed, for a lot of us in smaller companies, getting our clients to actually consider taking an agi…

For a team of 3 devs + 1 scrum master: - Backlog refinement/grooming: 1.5 to 2h, 1 dev + SM (+ PO) - Technical refinement: 1.5h, all 3 devs - Sprint review + retrospective + planning: 4h, all 3 dev + SM (+ PO)

Story generation is done by the PO, that's most of his job (though we split stories during backlog refinement if needed).

I don't know if it sounds like a lot (I feel like it could), but turns out it's actually efficient, every minute of each meeting is productive because we know exactly what we're going in for, how it will go and what you want to get out of it!

Re: Why Development Teams Struggle to Deliver on Time, on Budget, or at All

#234
When I was just starting out as a wet-behind-the-ears developer, an older, grizzled dev gave me invaluable advice for estimating: take your best guess of how long something will take and double it, that's what you tell the users. That advice has stood me in good stead, though I've found as I've gotten older and more experienced, I could probably reduce the factor from 2 to about 1.5.

Re: Why Development Teams Struggle to Deliver on Time, on Budget, or at All

#235
post #228

Earlier quoted context omitted.

Just to give an example of two projects I worked on at the same company within a few months of each other: The first involved taking all sorts of situational data from various sources in an airport, e.g. radar systems with their own binary formats, and combining them with the work of a postgrad researcher who had come up with an algorithm to optimise when pilots should turn on an aircraft's engine in order to save fu…

Aaargh20318 et al are making the claim that it is always inescapably like this and there's nothing you can do about it, because every day of programming is always fundamentally unlike anything you have ever done before. All I am suggesting is that this is an exaggeration. Doing something about it takes some commitment, however. Nobody estimates well without having practiced it, and been honest with themselves about w…

Understood, nothing to disagree with here.

Re: Why Development Teams Struggle to Deliver on Time, on Budget, or at All

#236
post #16

In my experience, deadlines set by higher-ups long removed from (or never having) the technical chops to be determining the deadlines in the first place. VP's and Directors are often the ones dictating the direction, which is great, but then also introducing deadlines, with helpful input from Directors who ALSO haven't touched code in many years. Generally that results in one of two things: * Product delivered on-tim…

Software is up there as the most complex thing ever created. People have been putting up houses for thousands of years, And modern big bridges for hundreds. Big software is ~40-60 years old. Its hard to have rules of thumb and best practice that will deliver the 'on time outcome'. Give it another 100 years. Most dev work right now is still just attempting to deliver correct software. let alone on time or on budget.

Nonsense. Software is not more complex as other engineering jobs. It is much less complex than e.g. architecture, which is a generalization over multiple technical jobs. In general you cannot trust a young architect with less than 20 years of experience. You can appreciate the end quality because of the ambition, but with the time and budget you need to be careful. But other typical engineering jobs also, which are not as simple and logical as software.

I worked professionally in a couple of high-tech engineering jobs, as architect, surveyor, city planner, civil engineering, construction, stage design, movies, automotive engineering (F1), games and at internet service providers. (but no airplanes, nuclear power plants or rockets, sorry. Lots of hospitals still.) The SW development jobs compared to that were always the easiest, most creative, most rewarding, and best paid also.

The problem with SW planning is pure lack of experience and poor engineering practices. A good SW engineer can hold his estimates, just as an experienced architect can hold his date and costs. In practice it rarely happens, I only know a couple of successful projects with smaller companies, but the good and big companies mostly deliver on time and budget.

As you mentioned houses and bridges. With houses you don't change engineering paradigms every 5 years as it might fancy you. You never try out new materials which are not tested over 10 years at least. These things need to stay up for 50 years at least. With bridges you cannot really plan them properly. Well, you can. But a typical bridge engineer multiplies the safety factor with 10, leading up to 10x more material as needed, as you can hardly estimate the dynamic peaks and resonance frequencies. With highly dynamic forces such as e.g. in a F1 gearbox, a huge spring with max 4000 Nm forces on your shafts. There you cannot apply any 10x rule. This thing needs to be perfect. You plan to avoid the peaks at all, avoiding the dangerous frequencies. Comparable to wind forces on a bridge. But in practice you look at best practices. You build dynamic physical models and simulations. And measure. And apply safety margins. E.g. with the big steel bridge next to my home, the famous "Blaues Wunder"in Dresden, the general public and politicians never trusted their engineers with this fancy new steel design, so it had to made 10x bigger and heavier, and even on opening day they invited the whole town to test it live, with lots of heavy trucks, a tramway and pedestrians walking over the bridge and jumping on it all at once. It stills stands today, but could have been made much simpler and elegant by todays knowledge.

Now compare that to modern SW engineering. You got a fancy new framework and language, looking for engineers with at least 10 years experience in that, with the framework existing for 2 years. You got massively overhyped tools which rarely deliver on their hype, complete lack of security, memory safety, type safety or concurrency safety, completely outdated API's like blocking IO or POSIX or threads, and undesigned monsters like C or C++. Good luck with that. That's of course a death march.

Still with proper tools and proper management it's much easier than everything else. In automotive everything was properly planned, with traditional waterfall. Everything worked as planned. Even if it was hugely complex, much more complex than a simple OS. In my own SW I never miss my milestones or overdo my budget. In architecture we always did. With huge costs up into billions. Only once we went under costs by 200 millions with the biggest building construction project in Austria. This was a miracle then.

Re: Why Development Teams Struggle to Deliver on Time, on Budget, or at All

#237
post #150

Software is design, not construction. This is why it's hard to estimate. You ask an architect to design a new skyscraper in a fixed three-month timeframe, good luck. It won't be what you want, or it'll have severe problems discovered during construction. Software developers are creative workers, we have to accept this. Unless you're doing the nth iteration of basic CRUD/RESTful web app, or a trivial "display data fro…

Architects regularly design new skyscrapers on fixed schedules. Look at some of the major developments going up in China.

Having built such a modern scyscraper in China by myself, I have to disagree. The first such scyscrapers where incredibly poorly planned, because they didn't have the proper budget and management culture. They probably outsourced the CAD work to some shop in Singapure or Indonesia, struggled with the metric/imperial system and tolerances. Workers conditions were horrible. People died all over, nobody cared, the bodies were just carried away. They did emergency welding of parts not fitting together on site on the 30th floor, and so on. The higher the less it fit. They didn't care about proper procedures. It was a nightmare.

Only when the Chinese realized it doesn't work this way, they gave the next projects to established european companies with experience, which didn't have to outsource their planning and construction, everything started to work out properly. Nowadays I'm incredibly proud of the changed Chinese engineering culture. They've surpassed Japan in all measures.

Re: Why Development Teams Struggle to Deliver on Time, on Budget, or at All

#238

Higher up manager here. I fully accepted the #noestimates movement and it is a complete blessing for all the teams and organizations I've implemented it in. Roast me.

Did you do that in an agency as well? I mean it's one thing if you are developing a product, but telling your clients "the project is finished when it's finished and we won't be able to tell you how much it will cost you until it's finished" doesn't really fly in my experience.

Me personally: no. But friends of mine owning an agency do this with all their clients. The gist of it is: don't think project, but iterations that deliver value every time you end one. The customer pays per iteration.

Re: Why Development Teams Struggle to Deliver on Time, on Budget, or at All

#239
post #151

There's a lot of snake oil for sale if you are looking to spend money on solving this problem; this probably is more of that. The reality is that mostly we've gotten better at avoiding things that clearly don't work or are historically obviously misguided/inappropriately expensive (waterfall, CMM level 5, etc); and instead emphasizing things that don't work slightly better: managing risk by doing iterations, not atte…

CMM level 5 does work, if you actually do it instead of just treating it as a paperwork exercise to satisfy an auditor. There's nothing in CMM that's contrary to or incompatible with Scrum or other agile methodologies. The major emphasis in CMM is on documenting your process (whatever that process is), training people properly, and continuously improving.

Yeah it works. It's just not very appropriate in a lot of situations because it is insanely expensive because it adds a lot of non functional bureaucracy and time consuming activities to the critical path of delivering software.
Post reply on HN