Live data from Hacker News

Software effort estimation is mostly fake research

shape-of-code.coding-guidelines.com

251–260 of 318 posts

Re: Software effort estimation is mostly fake research

#251
It is possible to deliver most software on time, but you need to be very clear about scope and unknowns and keep the units of work small.

The broad strokes of the design work need to be done before the estimate (i.e. gather requirements, identify constraints, decide on direction and tools).

If work is unknown or in large chunks, it needs to be broken up into smaller well-understood chunks before estimation.

When new ideas arise, they should usually be kept for future work.

When work is added, other work needs to be removed or the estimate revised.

If the work runs late, something should be removed from the scope to keep it on time, then done after delivery.

If work involves integration with other systems this is much harder unfortunately, but for some classes of software (much business software), it is not so hard to estimate.

Re: Software effort estimation is mostly fake research

#252
post #127
post #98

Earlier quoted context omitted.

The issue is that you are being asked to estimate something that has never been done before. Even houses always go over time and money and that is fairly straight forward. These days, I only give estimates in terms of units but without numbers. Hours, days, weeks, months, quarters or years. Some relatively small number of those units. If you want a quote it will take an extra 1/4 of the estimate worth of time for an…

> Even houses always go over time and money and that is fairly straight forward. This is an incredibly important point. I was "raised" in commercial software at a time when there was a lot of condescension internally towards ourselves as an industry because we couldn't do estimation and planning, and building was the favorite comparison. A $100 million building couldn't run late or over budget. That would be unimagin…

From what I've heard, in construction most of the delays happen with the foundations.

Everything that's below the ground is hard to predict, because you seldom know exactly what's in the ground.

Once your construction has progressed above ground, it's somewhat more predictable.

(But delays are still possible.)

Re: Software effort estimation is mostly fake research

#253

Earlier quoted context omitted.

It doesn’t even have to be a big construction project to hit budget and schedule issues. Rebuilding our house had us going right back to the drawing board having to get new plans drafted and approved. It worked out much better in the end but the unforeseen cost and time overrun was quite anxiety inducing at the time. But yeah the idea anyone has magically solved accurately predicting the future is completely bonkers.

I like to remember "plans are useless but planning is indispensable". The plan is useless because of the challenge of predicting the future. But the planning is useful because trying to predict the future gets you asking good questions.

More than that we need to be flexible to adjust a plan. You’d be amazed at how many people who quote that aphorism aren’t actually flexible. At least with a building they are also motivated by not leaving a physical monument to their failure.

Re: Software effort estimation is mostly fake research

#254

I am 20 years in development business now. This simple rule of thumb works for me and the team: (Your honest and concise estimation) * 3 There are just to many unknowns you cannot foresee. Software development is complex.

Great anecdote posted here a few years ago: https://www.quora.com/Why-are-software-development-task-esti...

Re: Software effort estimation is mostly fake research

#255

I seriously don't understand the obsession with estimates in the paradigm it is pushed in. Want to be steady at a reliable pace, work for a few months and figure out the average and deviations. Need a priority on tasks, use a priority queue using whatever method you assign it by, often a combination of deadline, value added, complexity and more. Yes, I get on a high level, one needs to be able to say "yes we can make…

That is absolutely what it is. For example, people are comparing favorably to how sales reps are now forced to record everything they do in something like Salesforce to try to prove they are productive, which by all accounts is pretty awful for the actual people doing the selling and doesn't actually improve sales--it just makes management feel like they're doing something.

Re: Software effort estimation is mostly fake research

#256
post #82

The issue with estimates are expectations. While nobody acknowledges it, you're not actually asked for an estimate, you're being asked for a quote. The difference is when you're asked for a quote, you're asked how much you will be charging, with the expectations that you'll be willing to eat into your own margins to give a lower quote. That's why it's a negotiation, where you negotiate how much extra effort, time and…

And of course once the dev team realises that their "estimates" are actually being treated as quotes, and that their ability to find time to vacation, sleep, or even see their families will be contingent on their quote being sufficiently padded...

Now you have a recipe for a highly adversarial relationship between management and engineering throughout the planning and development process.

Re: Software effort estimation is mostly fake research

#257

Earlier quoted context omitted.

Didn't Joel Spolsky implement it all the way back in the stone age... er, 2007[0]? The idea is old. The problem is, in many software companies, none of the tasks look alike. So your past data is going to be somewhat dirty, particularly at team level. Perhaps as an individual, intimately familiar with the problems you encounter, being able to introspect your own thought process - perhaps you or me, we could learn to e…

> The problem is, in many software companies, none of the tasks look alike. So your past data is going to be somewhat dirty, particularly at team level Tasks don't really give you anything though. I think if it was measured in terms of features, that would work a lot better. And with a Monte Carlo simulation, you get to know the time it would take you to complete any random feature on any random period. So maybe you…

Monte Carlo here is just a way of adding together estimate distributions without doing any of the fancy math. The variance of resulting compound estimate strongly depends on the variance on inputs.

Feature-based estimation seems to coarse to me. Not only none of the features is ever like the others once you dig into it, they're also interconnected.

Re: Software effort estimation is mostly fake research

#258
post #127

Earlier quoted context omitted.

> Even houses always go over time and money and that is fairly straight forward. This is an incredibly important point. I was "raised" in commercial software at a time when there was a lot of condescension internally towards ourselves as an industry because we couldn't do estimation and planning, and building was the favorite comparison. A $100 million building couldn't run late or over budget. That would be unimagin…

I've seen a lot of industries compared to programming, like construction and manufacturing. For example, people trying to apply lessons from the Toyota Production System to programming. This doesn't make sense, because we have systematized the construction of executable programs with extremely high reliability and predictability: we call it "Copy File" It's the design that is the hard part, and the process of coding…

This point was recognized quite early. there is an influential article by Jack W. Reeves from 1992 [1] that looks at the manufacturing analogy and argues that "the code is the design" whereas the "manufacturing" is just the compilation and packaging (that is, super cheap and automated). I believe it‘s this line of thinking that inspired Agile practices, where software developers are seen as engineers rather than code monkeys.

[1]: https://www.developerdotstar.com/mag/articles/reeves_design....

Re: Software effort estimation is mostly fake research

#259
Hasn't anyone tried to replace time estimates with probabilities?

(Like "There's a 80% chance we'll finish this today." ) You immediately start to understand why it can take so much longer than expected (You've rolled a dice before; I want a six... How long will it take)

Re: Software effort estimation is mostly fake research

#260
post #82

The issue with estimates are expectations. While nobody acknowledges it, you're not actually asked for an estimate, you're being asked for a quote. The difference is when you're asked for a quote, you're asked how much you will be charging, with the expectations that you'll be willing to eat into your own margins to give a lower quote. That's why it's a negotiation, where you negotiate how much extra effort, time and…

There may be a lot of uncerntainty in Data Science and ML projects. However, recently I started feeling like I actually have it better than someone from pure software engineering sides of things:

For either, there is often a function from time spent to quality. 100% perfection is basically impossible and before that the function increases very slowly, seemlingly logarithmicly.

For SWE, expectations are often close perfect solutions. Too greedy effort estimations cause a lot of trouble. For DS/ML, however, perfect is usually off the table and this fact is widely (not universally though) accepted. When it is accepted to give estimates in this way, suddenly there no harm from being quoted on it and I really don't mind to give estimates anymore, where I just make a guess at a good 80/20 point. If I am wrong with that point, chances are nobody on the outside/higher up ever knows.

This may be different in domains where very clear targets have to be met (e.g., "self driving cars that pass lawmaker's requirements for use on the streets") and then I'd guess it is a true nightmare.

Like this, I never felt overly pressured by ML/DS deadlines over the last years. Some things were great successes, sometimes the quality wasn't great enough and projects were stopped or customers left. But there never really was a case where anyone thought that working extra long might have been an option to meet higher expectations.

I don't really have a solution for SWE, I don't really see how one would sell something like "I can do it in X time and it will only crash / stop working / make mistakes / become too slow / have vulnerabilities so often. More time will lead to fewer problems". This just isn't what's expected. But at least for complex systems and security vulnerabilities, I'd argue it is actually quite true. Guarantees for 100% perfection just aren't realistic. Avoiding the most obvious pitfalls is done rather quickly and the more time spend, the more is needed for further improvements.

Post reply on HN