Software effort estimation is mostly fake research (2021)

If you want to understand estimations and projects, you have to study them. This is a social science project. It's not a math game. It's not a problem that you can solve by collecting oversimplified numbers from dubious sources based on incoherent definitions of the units of measurement.

If somebody reports that they spent x days developing a component, what does that mean? What were they doing during those days? How do we know what they were doing? We don't know unless a researcher was right there in the room taking notes in real-time. There are so many confounding variables involved, even if your data collection is perfect, that analysis of the data is a process fraught with opportunities for accidental overfitting or researcher bias.

Re: Software effort estimation is mostly fake research (2021)

#6

Post by awb » Fri, Jun 16, 2023, 1:02 AM UTC

Ran a successful digital agency for years.

Estimation was mostly guesswork. There’s a theory that work expands to fill the time you give it and I found that to be largely true.

Other agencies would quote 3 months for projects we’d complete in 6 weeks. Even then I was taking engineers 3 week estimates and doubling them for buffer.

We didn’t do anything special other than commit individuals to a single project, or two at most.

We would of course add additional resources if necessary to hit deadlines and then adjust future estimates as needed.

After a few years estimation felt more instinctual than scientific.

Re: Software effort estimation is mostly fake research (2021)

#7

Post by readthenotes1 » Fri, Jun 16, 2023, 1:04 AM UTC

satisfice wrote: post #5

The datasets are not even data in the first place. If you want to understand estimations and projects, you have to study them. This is a social science project. It's not a math game. It's not a problem that you can solve by collecting oversimplified numbers from dubious sources based on incoherent definitions of the units of measurement. If somebody reports that they spent x days developing a component, what does tha…

The one bit of project estimation research that I actually believed was that project estimates are closely affected by timeline desires of major finders/"stakeholders".

Anecdata:

I once asked a VP of a major telecom why the project deadline was Sept 30 and not Oct 1 or Sept 28, and he replied "because my quarterly bonus depends on it"

Re: Software effort estimation is mostly fake research (2021)

#8

Post by michelpp » Fri, Jun 16, 2023, 1:16 AM UTC

awb wrote: post #6

Ran a successful digital agency for years. Estimation was mostly guesswork. There’s a theory that work expands to fill the time you give it and I found that to be largely true. Other agencies would quote 3 months for projects we’d complete in 6 weeks. Even then I was taking engineers 3 week estimates and doubling them for buffer. We didn’t do anything special other than commit individuals to a single project, or two…

> There’s a theory that work expands to fill the time you give it

https://en.m.wikipedia.org/wiki/Parkinson's_law

Re: Software effort estimation is mostly fake research (2021)

#9

Post by Solvency » Fri, Jun 16, 2023, 1:26 AM UTC

michelpp wrote: post #8

awb wrote: post #6

Ran a successful digital agency for years. Estimation was mostly guesswork. There’s a theory that work expands to fill the time you give it and I found that to be largely true. Other agencies would quote 3 months for projects we’d complete in 6 weeks. Even then I was taking engineers 3 week estimates and doubling them for buffer. We didn’t do anything special other than commit individuals to a single project, or two…

> There’s a theory that work expands to fill the time you give it https://en.m.wikipedia.org/wiki/Parkinson's_law

It can also just barely squeeze into the time you give it, by the skin of your teeth.

Re: Software effort estimation is mostly fake research (2021)

#10

Post by quickthrower2 » Fri, Jun 16, 2023, 1:40 AM UTC

My instinct is that if you can put numbers and predictions based on real world estimations, then you are estimating simple things.

For example, you can estimate how long it will drive from A to B. Google maps does a brilliant job of this, showing the busy sections. Often I get there +/- a minute from what is estimated over a 60 minute journey.

But estimating how long someone will take to do a project, where the project by definition is something new in the context of the people doing it. If it wasn't then the code would already exist, and the estimate will be "how long to deploy that thing you already have using CI".

So in software you are always doing something new. We are building Sydney Opera Houses, not prefab housing estates.