Why are software development estimates regularly off by a factor of 2-3? • Hacker News Board

Re: Why are software development estimates regularly off by a factor of 2-3?

#151

Post by ohyes » Sat, Nov 30, 2013, 2:06 PM UTC

gnarbarian wrote: post #117

wfunction wrote: post #109

Earlier quoted context omitted.

For what it's worth, I haven't seen hardware engineers complaining about this issue as much as software engineers, which I find baffling. I think if you're going to justify why software development is such a "hard" problem, you should mention what makes it so different from other fields.

Hardware engineers have more clearly defined requirements, It's also well understood by everyone that you can't just change the design halfway through production with no additional cost. Hardware engineers clients are mostly other engineers who know their shit very well. Hardware engineering is also more formalized with a larger percentage of engineers actually possessing relevant degrees with common well established…

8 and 3 are the biggest in my opinion. 3 should have it added that the we are bad at getting the requirements out of customers, then writing it down and sticking to what is written.

8 isn't just about confrontation, the person who says 'yes' is more likely to not be fired than the person who says no. You need to frame all of those times you say no as alternatives, not as flat saying no.

Re: Why are software development estimates regularly off by a factor of 2-3?

#152

Post by _wwz4 » Sat, Nov 30, 2013, 2:36 PM UTC

gcv wrote: post #90

_wwz4 wrote: post #18

So the article tries to describe the hidden complexity of software with a hiking analogy but then it leaves a huge part missing. Let's extend the analogy somewhat... this isn't your first hiking trip. In fact, you've been on dozens and dozens of hiking trips. You're an experienced hiker. Yet, why do you continue to give meaningless as-the-crow-flies estimates of how long it will take you to get to your destination? T…

It's more like this: you've been on dozens and dozens of hiking trips. Except these trips generally go through seismically unstable territory and war zones. You just never know when you'll have to dig a tunnel underneath a battlefield to get to the next river you'd have to cross — a river unmarked on your map, or perhaps the result of some aggressive "engineering" by someone since your map was drawn. Then you get an…

Meh, you're just contriving extreme circumstances for your analogy so that the software developer is both a super hero and has no blame in being late with deliverables.

There are simple methods to combat gross inaccuracies in the client specification. You document the requirements for what you'll be delivering in a carefully worded Statement of Work that the client signs off on, you ask to be paid in chunks as you go with a portion up front, and you implement a system of change control so that it's totally clear if the customer is sending you off in a completely new direction. At that point, you book what you've worked at your hourly rate and you submit new work and time estimates for the changes.

Re: Why are software development estimates regularly off by a factor of 2-3?

#153

Post by crusso » Sat, Nov 30, 2013, 2:41 PM UTC

sandGorgon wrote: post #134

crusso wrote: post #39

Earlier quoted context omitted.

Developers, in my experience, are quite often asked to leave the company after such estimates. I've seen developers let go because they weren't very good at writing software. I've seen developers let go because they were painfully anti-social to the point that they were negatively impacting the rest of the organization. I have never ever seen a developer let go because their estimates were crappy.

well maybe it doesnt happen at a "developer" level, if that's what the management hierarchy calls it. But it definitely does happen on an engineering manager level and is actually fairly common in the VP Engineering role.

Right, I've seen that and I'm okay with it. At the individual developer stratum, you expect the level of naivete that was in the original article we're discussing.

At the VP level, you're expected to understand what the Developers and Development Managers don't know about the SDLC. You're expected to bridge the gap in their lack of understanding by instituting processes and controls while mentoring them to become better at what they do so that the organization can succeed.

Re: Why are software development estimates regularly off by a factor of 2-3?

#154

Post by thirdtruck » Sat, Nov 30, 2013, 3:06 PM UTC

ataylor32 wrote: post #54

thirdtruck wrote: post #38

Earlier quoted context omitted.

Have you shared these spreadsheets and lists online? They sound like something that could turn into an excellent, three-figure or more information project.

I, too, would be interested in seeing these. Here are some guidelines from somebody else that I found helpful, though: http://www.missionfamilybank.org/software-estimating/

Thanks!

Re: Why are software development estimates regularly off by a factor of 2-3?

#155

Post by shitgoose » Sat, Nov 30, 2013, 3:18 PM UTC

If only they used one of those frameworks, that lift you off the ground and glide you towards the destination effortlessly, graciously pulled by flying pigs, there would be no delay. They would even have time for daily scrums. Always pick the right technology stack, guys!

Re: Why are software development estimates regularly off by a factor of 2-3?

#156

Post by finnw » Sat, Nov 30, 2013, 3:37 PM UTC

analog31 wrote: post #121

pratik661 wrote: post #114

Earlier quoted context omitted.

Its much easier to justify changes in software requirements vs changes in hardware requirements because the perceived cost of adding a few more features on your application is low. Compare that to say.. adding another register to your chip..which might add millions to the whole production process. The changes can be made while the app is live. You can not do that for hardware.

I've seen the opposite, where the effort and bureaucracy to change the software is so prohibitive that changes are made to electronics or even mechanics to avoid a software change.

> ... to avoid a software change.

"It's OK, it doesn't need any code changes, just a modification to the configuration files, so the approval process does not apply."

That company's developers just need to learn the trick of making the configuration file format Turing-complete.

If you don't have time to write your own interpreter, XSLT is an easy one to sneak by PHBs.

I've seen one system where 70-80% of the functionality was implemented in XSLT for this reason. It meant changes could be delivered in 5 days instead of 35. Obviously good news for the customer. The PHBs were pleased that the bug count in the C++ core had dropped (it hadn't really, we just stopped using the buggy modules.) The integration testers were pleased that the end product worked. Basically everyone was happy except the C++ reviewers in QA.

Re: Why are software development estimates regularly off by a factor of 2-3?

#157

Post by michaelt » Sat, Nov 30, 2013, 3:57 PM UTC

wfunction wrote: post #109

tmoertel wrote: post #64

Software-development estimates are regularly off by a large margin for two reasons. First, the problem is inherently hard. Second, social and business pressures bias the estimates downward. Why is the problem inherently hard? To see why, let's imagine that the All-Knowing Fairy Godmother of Software Estimation descends from the heavens and lends to us her magic estimating function F that, applied to any software proj…

For what it's worth, I haven't seen hardware engineers complaining about this issue as much as software engineers, which I find baffling. I think if you're going to justify why software development is such a "hard" problem, you should mention what makes it so different from other fields.

Engineer with hardware and software experience here.

Software is held to much lower engineering standards because it can be made much more complicated, by most measures of complexity; and complexity in hardware drives up development and per-unit costs, whereas complexity in software drives up upfront costs only, so most people push complexity into software.

Consider an all-mechanical watch. Even a watch that just accurately displays time and day of week is going to be fairly complicated. If you want the day of month to account for the length of different months, that's more complicated again, and if you want to account for leap years, especially the mod-100 years that's more complicated again. To say nothing of products like the Calibre 89 [1].

A product like Pebble will do all that without batting an eyelid, and a hundred other things that would be impossible to do mechanically. Setting the time automatically. Programmable watch faces. Scheduling meetings and synchronizing with other watches, phones and computers. Correcting across time zones and daylight savings transitions. Accessing real time train schedules and weather.

[1] https://en.wikipedia.org/wiki/Calibre_89

Re: Why are software development estimates regularly off by a factor of 2-3?

#158

Post by cuillevel3 » Sat, Nov 30, 2013, 4:09 PM UTC

I think the calculation usually goes like that:

  * Take a day to come up with "estimates" for the project,
    can't bill that time of course.
  * Sell fantasy gant chart to the customer.
  * Take 10-20% of the projects time away for "project management".
  * Actually develop the software.
  * Fix "bugs" until customer goes silent.

If you want good estimates you need to take estimates and project management more serious and get away from that waterfall methodology.

Re: Why are software development estimates regularly off by a factor of 2-3?

#159

Post by Philadelphia » Sat, Nov 30, 2013, 4:30 PM UTC

avmich wrote: post #19

ssiddharth wrote: post #6

I admit it's funny but if I showed this to a non-programmer friend, he'd be asking why I didn't research my route better or start on an unknown trail with minimal planning.

Because the research will take a significant part of the project time. Once the thorough research is done - third-party libraries are verified (that they do what's needed, and not too buggy), prototypes are written (to confirm libraries and internal dependencies work as expected), proof-of-concepts are run (to actually see that it's possible for a reasonable effort), time is measured (to confirm the way to go), inter…

Maybe part of the problem is that development organizations don't tend to charge for pre-estimate planning.

Re: Why are software development estimates regularly off by a factor of 2-3?

#160

Post by wildgift » Sat, Nov 30, 2013, 5:20 PM UTC

knappador wrote: post #102

wpietri wrote: post #44

Earlier quoted context omitted.

I'm glad you got better at it, but I don't think software estimates can ever be very good. Nor should they be. That a project is estimatable has to mean that novelty is low and predictability is high. But low novelty is a sign of duplication, which should be factored out into things like libraries, frameworks, and self-service tools. High predictability is either a sign of duplication or it's a sign that people have…

I do flat-rate projects like this using a library that I have to fill in gaps myself. Even in the context of very high unpredictability, I have to make good predictions else suffer low compensation exactly like the parent poster did. I like crow-flies vs hiking analogy. I'd like to add upon it for so-called "green field" programming as it relates to getting cost predictions correct and understanding programmer time g…

I used to estimate small websites (The big unknown - and the highest risk - was managing integration between external systems, or doing complex integration between internal systems. Anything that might benefit from aspect-oriented programming, like access control lists, or a more elaborate front-controller, or universal exports in json format, or a new paradigm like AJAX, would be difficult to estimate.

That fits your breakdown of projects into linear parts, and exponential parts.

Anyway, my estimates were good, but my business management was not - I was always in learning mode because I never studied business and had no experience. So I went W-2 and went into a different field because an opportunity to learn politics emerged. What's interesting is that successful political campaigns are entirely about estimating and budgeting, in real time more or less - you spend money and see results day by day, or after a couple weeks. It's driven by statistics (and messaging). I just watch from the sidelines, and don't do politics, but I think we could learn something from it. It's more like agile XP than waterfall.

The Obama campaign was XP, by necessity. The Healthcare.gov website thought it was waterfall... again by necessity.