Software effort estimation is mostly fake research • Hacker News Board

Re: Software effort estimation is mostly fake research

#241

Post by morelandjs » Tue, Jan 19, 2021, 4:05 AM UTC

My issue with effort estimates is that it is taboo to report estimates with uncertainty (think how silly that sounds). I'll often give a low-end / high-end and the person asking for the estimate will tell me they want a single number. If you did this in physics you would get laughed and for good reason. It's bad science, and it's bad forecasting.

You should never claim better precision than you know. It's often more accurate to say "more than an hour but less than a week" than to say "6 hours". We learn scientific precision in middle school... why is modern workflow management still so bad at uncertainty propagation?

Re: Software effort estimation is mostly fake research

#242

Post by newscracker » Tue, Jan 19, 2021, 4:24 AM UTC

So research into software estimation also seems to follow Hofstadter's Law: “It always takes longer than you expect, even when you take into account Hofstadter's Law.”

Re: Software effort estimation is mostly fake research

#243

Post by nautilus12 » Tue, Jan 19, 2021, 4:49 AM UTC

BowBun wrote: post #100

It's unfortunate that this HN thread has been reduced to the generic discussion about software estimates when the article is specifically talking about research done on the topic of software estimates. According to the article, proper research remains a struggle due to outdated datasets from before modern agile methodologies, and that the modern datasets from industry are hard if not impossible to gather. If industry…

Dont be so hard on HN, I would say given how niche the actual article topic is, the more generic discussion about software estimation is both relevant and relatable.

Re: Software effort estimation is mostly fake research

#244

Post by kanzenryu2 » Tue, Jan 19, 2021, 4:51 AM UTC

durandal1 wrote: post #21

jakubp wrote: post #2

If someone can conclusively teach inexperienced programmers good approach to estimates (methodology) + help embed this into sales process of a software house-type company, I know some folks who'd love to have this :) My own experience has been this: people make estimates, client has expectations based on some variant of those, and something later happens but so much change is introduced during the actual software dev…

Apple executes massive feature releases on a yearly waterfall-like schedule across hundreds of teams. It requires honesty, transparency, strong leadership (ruthless prioritization), strong cross-team goodwill, strong cross-team collaboration as well as a solid body of experienced top-tier engineers. Other than that there's no magic. While there are teams within apple that does scrum/agile, none of the core OS/framewo…

The macbook range had years of delays

Re: Software effort estimation is mostly fake research

#245

Post by galaxyLogic » Tue, Jan 19, 2021, 5:12 AM UTC

wheelie_boy wrote: post #210

dkarl wrote: post #127

Earlier quoted context omitted.

> Even houses always go over time and money and that is fairly straight forward. This is an incredibly important point. I was "raised" in commercial software at a time when there was a lot of condescension internally towards ourselves as an industry because we couldn't do estimation and planning, and building was the favorite comparison. A $100 million building couldn't run late or over budget. That would be unimagin…

I've seen a lot of industries compared to programming, like construction and manufacturing. For example, people trying to apply lessons from the Toyota Production System to programming. This doesn't make sense, because we have systematized the construction of executable programs with extremely high reliability and predictability: we call it "Copy File" It's the design that is the hard part, and the process of coding…

> It's the design that is the hard part

Design but also requirements. You don't really know what should be built until you have a prototype. And estimating how long it will take to come up with a prototype that does what you think it should do, in other words coming up with requirements, is just the snake eating its tail.

Re: Software effort estimation is mostly fake research

#246

Post by galaxyLogic » Tue, Jan 19, 2021, 5:16 AM UTC

TeMPOraL wrote: post #178

gugagore wrote: post #169

Earlier quoted context omitted.

I like to remember "plans are useless but planning is indispensable". The plan is useless because of the challenge of predicting the future. But the planning is useful because trying to predict the future gets you asking good questions.

Also because planning gets you to make your desires explicit and to put them in a useful framework. Unexpected problems, disruptive as they may be, don't usually derail the whole plan. Some parts may need to be rethought, but others are still valid, and the whole thing keeps you focused.

Programming is isomorphic to theorem proving (according to Curry-Howard).

Therefore trying to estimate a software project is a bit like trying to estimate how long it will take to prove a new theorem in mathematics, assuming you would know which exact theorem you want to prove.

Re: Software effort estimation is mostly fake research

#247

Post by ChicagoDave » Tue, Jan 19, 2021, 5:23 AM UTC

Unfortunately, the business folks insist on making up arbitrary dates and costs. It’s the way of the world.

Re: Software effort estimation is mostly fake research

#248

Post by galaxyLogic » Tue, Jan 19, 2021, 5:28 AM UTC

I think the problem is that if you have a timeline you can produce the result within time and budget, but at the expense of quality.

The time and cost estimates do not measure the amount of technical debt in the outcome. it may good great on paper but that is because of a vaguely defined specification which says nothing about the quality, including amount of te4chnical debt (meaning mostly maintainability).

Re: Software effort estimation is mostly fake research

#249

Post by aidenn0 » Tue, Jan 19, 2021, 5:49 AM UTC

snarf21 wrote: post #98

didibus wrote: post #82

The issue with estimates are expectations. While nobody acknowledges it, you're not actually asked for an estimate, you're being asked for a quote. The difference is when you're asked for a quote, you're asked how much you will be charging, with the expectations that you'll be willing to eat into your own margins to give a lower quote. That's why it's a negotiation, where you negotiate how much extra effort, time and…

The issue is that you are being asked to estimate something that has never been done before. Even houses always go over time and money and that is fairly straight forward. These days, I only give estimates in terms of units but without numbers. Hours, days, weeks, months, quarters or years. Some relatively small number of those units. If you want a quote it will take an extra 1/4 of the estimate worth of time for an…

If you give a bonus for getting done on time/early, construction will finish pretty darn close to the estimates. Construction is underbid at least partly because the incentives are setup for underbidding.

A lot of the rest is Planning fallacy[1], particularly with individual contractors, but larger contracting businesses can absolutely come up with realistic estimates based upon past experience.

1: https://en.wikipedia.org/wiki/Planning_fallacy

Re: Software effort estimation is mostly fake research

#250

Post by kevinmchugh » Tue, Jan 19, 2021, 6:08 AM UTC

cmroanirgo wrote: post #13

jakubp wrote: post #2

If someone can conclusively teach inexperienced programmers good approach to estimates (methodology) + help embed this into sales process of a software house-type company, I know some folks who'd love to have this :) My own experience has been this: people make estimates, client has expectations based on some variant of those, and something later happens but so much change is introduced during the actual software dev…

The problem is knowing how fast they learn. For a n00b this is basically impossible, and takes cost monitoring to notice their rate of learning. For a person with the basics of the language that they'll be using, and a tiny bit of experience in the libraries they'll be using, things get easier. For everyone (n00b to senior) I would always recommend designing & breaking things down into no smaller 1/2 day blocks, wher…

It's very useful to break tasks down so instead of a month long "upgrade elasticsearch" story, there's dozens of day-or-less tasks to work on. Often it's not noticed how much effort goes in to creating these new smaller tasks. If you aren't extremely familiar with the targeted version of elasticsearch, you'll have to spend significant amounts of time investigating, planning, and writing.

You can get very high accuracy on estimates by spending lots of time creating smaller stories and encouraging people to over estimate. This may not be in the best interest of the project or the business.