Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Software effort estimation is mostly fake research

shape-of-code.coding-guidelines.com

Post reply on HN
Print view View original

141–150 of 318 posts

Re: Software effort estimation is mostly fake research

#141

Post by benhurmarcel »

koonsolo wrote: post #136
didibus wrote: post #82

The issue with estimates are expectations. While nobody acknowledges it, you're not actually asked for an estimate, you're being asked for a quote. The difference is when you're asked for a quote, you're asked how much you will be charging, with the expectations that you'll be willing to eat into your own margins to give a lower quote. That's why it's a negotiation, where you negotiate how much extra effort, time and…

I once had a manager that thought he could negotiate estimates. That is like negotiating with the weatherman about the weather. Sure, maybe in the end you can convince them it will be finished sooner. But the fact is that it really doesn't change reality.

I find it much less stressful to deal with since I've made peace with the fact that some managers want to be lied to.

Re: Software effort estimation is mostly fake research

#142

Post by disgruntledphd2 »

djoldman wrote: post #93

>Those estimation datasets that were flogged to death in the 1990s using non-machine learning techniques, e.g., regression. >...non-machine learning techniques, e.g., regression. Is this where we are now? The connotation of "machine learning" doesn't include regression? Wow.

They (almost certainly) mean linear regression, which many people appear to regard as statistics and boring, as opposed to single layer neural networks, which are cool.

To be fair, this is a pretty common perspective in computer science academia and adjacent fields (and probably occurs in a lot of fields where people don't focus on statistical learning..ahem I meant ML).

Re: Software effort estimation is mostly fake research

#143

Post by lixtra »

koonsolo wrote: post #136
didibus wrote: post #82

The issue with estimates are expectations. While nobody acknowledges it, you're not actually asked for an estimate, you're being asked for a quote. The difference is when you're asked for a quote, you're asked how much you will be charging, with the expectations that you'll be willing to eat into your own margins to give a lower quote. That's why it's a negotiation, where you negotiate how much extra effort, time and…

I once had a manager that thought he could negotiate estimates. That is like negotiating with the weatherman about the weather. Sure, maybe in the end you can convince them it will be finished sooner. But the fact is that it really doesn't change reality.

Typically there are many dimensions to a solution, the negotiation can be about trading one dimension for another.

- does it have to be instant or can we run it as a batch

- maybe it’s enough to have this tool in English only

Etc.

Re: Software effort estimation is mostly fake research

#144

Post by aidos »

cgrealy wrote: post #75
a_wild_dandan wrote: post #58

Earlier quoted context omitted.

You've perfectly nailed the fundamental issue here. For many software projects, you simply cannot make meaningful granular estimations for parts of them. It doesn't matter how many story-point poker sessions you hold. Some software work cannot be reduced to a positive integer. In our business scopes creep, bugs plague us, and myriad issues make precise estimations pointless. It's engineering Numberwang. I get that fo…

> It's engineering Numberwang. Can I just say that is the greatest description of software estimation I have ever read? I am 100% stealing that.

We use it internally ourselves. It’s from That Mitchell and Webb Look.

Re: Software effort estimation is mostly fake research

#145

Post by nikolaj »

Aldipower wrote: post #4

I am 20 years in development business now. This simple rule of thumb works for me and the team: (Your honest and concise estimation) * 3 There are just to many unknowns you cannot foresee. Software development is complex.

same experience level, similar formula. Take your gut estimate, double it, double that new value, then add another "increment".

e.g. say you think it will take a day, so 2 days, 4 days, add another day, likely estimate is 5 days.

My pet theory is that when we estimate, we typically think of how long it will take to figure out a working solution to the problem, but forget about how long it takes to debug it, add tests, rework for changed requirements and unexpected nuances, and then roll it out and do any training, etc.

Re: Software effort estimation is mostly fake research

#146

Post by k__ »

snidane wrote: post #137

Software development which is a repeatable and already defined process is totally possible to predict and estimate. Most tasks of repeatable processes follow normal distribution and is predictable. Deviations from expected mean will be due to predictable factors of the environment such as failed disk or sleepy programmer. You can apply arbitrary six sigma methodology to measure such process with accuracy. The problem…

"a repeatable process would be immediately automated away by writing a function, library, framework or any such tools"

This is the crucial point here.

Source code is a (almost) self-assembling blueprint.

The actual product that will be build is the software and software is a configuration of matter, in this case of a computer.

The source code/blueprint for a house is not self-assembling. Compiling such a blueprint requires you to configure the building materials in a way that they become a house.

With better robotics, we will probably get there at some point in the future.

And with software we will always be in a place where you either do new stuff the first time manually or with crappy tools the 100th time.

Re: Software effort estimation is mostly fake research

#147

Post by froh »

BowBun wrote: post #100

It's unfortunate that this HN thread has been reduced to the generic discussion about software estimates when the article is specifically talking about research done on the topic of software estimates. According to the article, proper research remains a struggle due to outdated datasets from before modern agile methodologies, and that the modern datasets from industry are hard if not impossible to gather. If industry…

> How large are these datasets that have attracted so many research papers?

> The NASA dataset contains 93 rows (that is not a typo, there is no power-of-ten missing), COCOMO 63 rows, Desharnais 81 rows, and ISBSG is licensed by the International Software Benchmarking Standards Group (academics can apply for a limited time use for research purposes, i.e., not pay the $3,000 annual subscription). The China dataset contains 499 rows, and is sometimes used (there is no mention of a supercomputer being required for this amount of data ;-).

> Why are researchers involved in software effort estimation feeding tiny datasets from the 1980s-1990s into machine learning algorithms?

> Grant money. Research projects are more likely to be funded if they use a trendy technique, and for the last decade machine learning has been the trendiest technique in software engineering research. What data is available to learn from? Those estimation datasets that were flogged to death in the 1990s using non-machine learning techniques, e.g., regression.

Is this telling me that most theories about "sw estimation best practices" are cargo cults o-O ?

Re: Software effort estimation is mostly fake research

#148

Post by edelans »

taeric wrote: post #89

My problem with this is all estimation is hard. Period. Quantitative discussions of something that has been done? Fairly easy, if still not accurate. Discussing something that hasn't been rehearsed before? You can really only discuss how long you are willing to work on it. Not how long it will take to get done. Fun examples. How long would it take you to clean out your fridge? How long would it take you to learn to p…

I like this analogy. But I was picturing myself asking my sales manager "How long would it take you to learn to play a song on piano?", and I'm pretty sure he would reply "but you touched a piano before, you are supposed to be a professional piano player ! A professional piano player surely knows how long it would take him to learn how to play a song". So I guess he would miss the point totally :/

Re: Software effort estimation is mostly fake research

#149

Post by encoderer »

Don't do an estimate, build a prototype.

Don't do an estimate, build a prototype.

Don't do an estimate... seriously... build a prototype.

And then throw it out.

And THEN do an estimate. It will probably be pretty accurate.

Not always easy to do this in a tech company but as an eng director I was able to get it done and it changed a seriously broken process based on multi-week scoping and estimation futility.

Re: Software effort estimation is mostly fake research

#150

Post by ConnorLeet »

wefarrell wrote: post #135
jakubp wrote: post #2

If someone can conclusively teach inexperienced programmers good approach to estimates (methodology) + help embed this into sales process of a software house-type company, I know some folks who'd love to have this :) My own experience has been this: people make estimates, client has expectations based on some variant of those, and something later happens but so much change is introduced during the actual software dev…

More important than estimating is knowing how to sequence the work. I often see teams try to build the entire DB schema first, or built the interface out one wire at a time. The agile approach of building small, vertical parts of each feature tends to work much better in my experience. For example say you have a CRUD application with a page for editing and searching complex objects. Rather than building the entire se…

Although when working like this there can be a "Good enough" inertia
Post reply on HN