Software effort estimation is mostly fake research • Hacker News Board

Re: Software effort estimation is mostly fake research

#301

Post by JamesBarney » Tue, Jan 19, 2021, 4:24 PM UTC

Frost1x wrote: post #233

randomsearch wrote: post #111

Earlier quoted context omitted.

I'd say the problem is more from the academic side. If good data isn't available, then academics should not be publishing papers on toy data. It's meaningless. The goal is not to publish papers but to advance science.

While I tend to agree, the research we environment demands this nonsense. Also, they have to start somewhere to set a record to get future funding. Should probably do more survey studies of what data exists and where the state of things are then suggest new studies and seek funding for it. Short of large and completely government funded development projects, I think it would be a struggle to get data. Few businesses…

Research at business schools manage to find all types of data that companies believe is way more integral to their success than information about their software estimation process.

These researchers are choosing to waste their own time and governments money, because it's easier to play with new toy machine learning models than go to networking event and befriend software VPs in order to convince them to give you their data.

Re: Software effort estimation is mostly fake research

#302

Post by JamesBarney » Tue, Jan 19, 2021, 4:25 PM UTC

avdlinde wrote: post #175

randomsearch wrote: post #111

Earlier quoted context omitted.

I'd say the problem is more from the academic side. If good data isn't available, then academics should not be publishing papers on toy data. It's meaningless. The goal is not to publish papers but to advance science.

Well, the metric is #papers and #citations to get funding. So yes that's true, ideally, but...

Relevant blog post - It's not the incentives it's you.

https://www.talyarkoni.org/blog/2018/10/02/no-its-not-the-in...

Re: Software effort estimation is mostly fake research

#303

Post by x0x0 » Tue, Jan 19, 2021, 4:35 PM UTC

imtringued wrote: post #297

x0x0 wrote: post #204

Earlier quoted context omitted.

Inter-team dependencies. Marketing pushes. Customer commitments. New sales dependent on certain features. Building features to increase TAM. Partner commitments for interoperability / joint deployments. Even presentations of your roadmap to customers and prospects to get them bought into your vision of where the product is going. There's tons of things that require broadly knowing when X is going to be done beyond ca…

If you create low quality estimates willy nilly then don't be surprised that you will break them.

The above are reasons a business needs estimates / rough delivery timelines. I'm not sure how my discussion of the value of estimates is relevant to the quality of them.

Re: Software effort estimation is mostly fake research

#304

Post by JamesSwift » Tue, Jan 19, 2021, 5:10 PM UTC

jakubp wrote: post #2

If someone can conclusively teach inexperienced programmers good approach to estimates (methodology) + help embed this into sales process of a software house-type company, I know some folks who'd love to have this :) My own experience has been this: people make estimates, client has expectations based on some variant of those, and something later happens but so much change is introduced during the actual software dev…

If someone can teach _any_ programmer how to estimate accurately they will literally print money. Its inherently a hard problem (I'd argue a siren song). To truly succeed you need to perform a massive stakeholder education on software development, hidden costs, and the highly dynamic nature of the beast.

Then you need to train the IT professionals to build the dynamic nature into the conversation and process. You dont quote a number, you give a range. You communicate confidence levels and uncertainty clearly and continuously. You update assumptions as new information comes available. You _push back_ when the uncertainty is frowned on.

It can be done successfully on a micro scale sure, by working on it within an organization and/or team. But its not scalable and its not a 5-step plan. Its hard.

Re: Software effort estimation is mostly fake research

#305

Post by Leherenn » Tue, Jan 19, 2021, 8:32 PM UTC

didibus wrote: post #119

jeffreygoesto wrote: post #116

Earlier quoted context omitted.

Get something ready for Christmas business? Get a car on the road? Finish the cruise ship in time? ...

> In the context of software development? While your examples I'm not sure are super relevant from a software development context. I think you're pointing out that the estimate would help you decide when you need to start so you are done before the "real life events that needs the thing". Is that correct? Like for example I'm not sure it's useful to learn that no, you will not be ready for the Christmas event. Or no…

I have seen two software examples first hand. Usually it's because a feature can have much more value at a very specific date.

The first one was a feature for one of the minor verticals of the company product. Basically, something that would be nice to have and would be done eventually, but not a priority. However, the big annual trade show for that vertical was coming up, and the company could have a slot for a conference/demonstrate a feature. Marketing/sales were pretty convinced the aforementioned feature could be a big hit at the show, and provide lots of exposure. So we were asked whether it was possible to have it ready by then. If we could, then it would have a lot of value and would be priority 1, otherwise back to the backlog because it would be much harder to advertise it outside of the show.

Second example was caused by the reduction in VAT in Germany last year. Having internationalisation/German translation all of the sudden became pretty valuable because that would help push even more products in Germany.

Re: Software effort estimation is mostly fake research

#306

Post by maerF0x0 » Tue, Jan 19, 2021, 9:42 PM UTC

someguydave wrote: post #70

maerF0x0 wrote: post #61

Earlier quoted context omitted.

> but it probably took four times longer to make, I doubt it. Luxury goods don't proportionately more cost much more to produce than commodity goods. Instead they often derive their value from perceptions and exclusivity. That is people perceive the object to be more value for their own reasons, or they are made more expensive as a proof point of exclusivity / conspicuous consumption https://en.wikipedia.org/wiki/Con…

Mercades cars do often have more features, but I also am skeptical of claims that they are of higher quality.

In my experience Mazdas have the best tech/features at various price levels, compared to the competition's offering of the same feature

Re: Software effort estimation is mostly fake research

#307

Post by kridsdale1 » Tue, Jan 19, 2021, 9:43 PM UTC

AuthorizedCust wrote: post #43

durandal1 wrote: post #21

Earlier quoted context omitted.

Apple executes massive feature releases on a yearly waterfall-like schedule across hundreds of teams. It requires honesty, transparency, strong leadership (ruthless prioritization), strong cross-team goodwill, strong cross-team collaboration as well as a solid body of experienced top-tier engineers. Other than that there's no magic. While there are teams within apple that does scrum/agile, none of the core OS/framewo…

How do you know they are thoroughly waterfall? How do you know they don’t use agile, time-boxing, etc.? Apple is so secretive, it feels like we know of little other than what they are confident about releasing. The things that weren’t done on a schedule compatible with a hardware release, how often are they simply postponed? And for those postponements that they do admit, how many are waved away with a creative expla…

I worked there 6 years. It is 100% Waterfall. Things that are too buggy by July 1st are delayed by 12 months.

Re: Software effort estimation is mostly fake research

#308

Post by kridsdale1 » Tue, Jan 19, 2021, 9:45 PM UTC

robertlagrant wrote: post #114

durandal1 wrote: post #21

Earlier quoted context omitted.

Apple executes massive feature releases on a yearly waterfall-like schedule across hundreds of teams. It requires honesty, transparency, strong leadership (ruthless prioritization), strong cross-team goodwill, strong cross-team collaboration as well as a solid body of experienced top-tier engineers. Other than that there's no magic. While there are teams within apple that does scrum/agile, none of the core OS/framewo…

How do you know it's waterfall-like? If you ship working software every 2 weeks and every 26th one you release, hooray, you have yearly software that's super likely to work well.

Apple produces a new build of iOS or MacOS every 24 hours. For the many years that I had to suffer through that, about 40% of those builds were dogshit and unusable. The public gets the 360th release, and hopefully by then the PMs have driven enough burnout in the engineers that it is mostly acceptable to users.

Re: Software effort estimation is mostly fake research

#309

Post by kridsdale1 » Tue, Jan 19, 2021, 9:49 PM UTC

cgrealy wrote: post #75

a_wild_dandan wrote: post #58

Earlier quoted context omitted.

You've perfectly nailed the fundamental issue here. For many software projects, you simply cannot make meaningful granular estimations for parts of them. It doesn't matter how many story-point poker sessions you hold. Some software work cannot be reduced to a positive integer. In our business scopes creep, bugs plague us, and myriad issues make precise estimations pointless. It's engineering Numberwang. I get that fo…

> It's engineering Numberwang. Can I just say that is the greatest description of software estimation I have ever read? I am 100% stealing that.

LET'S ROTATE THE SPRINT BOARD

Re: Software effort estimation is mostly fake research

#310

Post by galaxyLogic » Tue, Jan 19, 2021, 10:46 PM UTC

garethrowlands wrote: post #294

galaxyLogic wrote: post #246

Earlier quoted context omitted.

Programming is isomorphic to theorem proving (according to Curry-Howard). Therefore trying to estimate a software project is a bit like trying to estimate how long it will take to prove a new theorem in mathematics, assuming you would know which exact theorem you want to prove.

Er, isn't it just getting your program to typecheck that's equivalent to proving the theorem (by construction)? I tend to find this isn't the main consumer of my software budget :-)

In a way I think yes but just getting the type-check right would be like proving ANY mathematical theorem. It is very easy to prove mathematical theorems if you don't care what those theorems are.

The challenge in mathematics, and programming, I believe, is what ("lemmas") to prove so that you can then prove more things that are useful for engineering, science, etc. . What functions to write in your program so that you can get its "main function" to do something useful.

Add to that the fact that much of the "type-checking" with current practical programming languages still has to be done informally, to describe what we expect of the result and then prove (to yourself and the client) that the program does that.

A lot of the thought-process that goes on in programmers' heads is informal reasoning proving things to yourself, "If I make this change here then that thing there must have a correct value, and then ..."